Awaiting tasks from multiple objects

Question

I have an application that uses MEF to load plugins. All of these plugins conform to the following interface:

public interface IPlugin {
    Task Start();
}

All the methods are implemented as async: public async Task Start()

When the application is running I have an IEnumerable<IPlugin> property available with all the plugins. The question is basically how I can run all the Start() methods in parallel and wait until all methods are finished?

I know about Parallel.ForEach(plugins, plugin => plugin.Start()), but this is not awaitable and execution continues before all plugins are started.

The most promising solution seems to be Task.WhenAll(), but I don't know how to send in an unknown list of methods into this without adding some scaffolding (which seems like overhead).

How can I accomplish this?

Sergey.quixoticaxis.Ivanov · Accepted Answer · 2018-09-01 09:52:26Z

5

And here's a one-liner:

await Task.WhenAll(plugins.Select(p => p.Start()));

The plugins will run asynchronously, but not in parallel. If you want for some reason to dispatch plugins to thread pool explicitly, you may add Task.Run with async lambda into Select.

edited Sep 1, 2018 at 9:52

answered Sep 1, 2018 at 8:34

Sergey.quixoticaxis.Ivanov

1,43513 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

TheHvidsten Over a year ago

Son of a bench, it's been staring me in the face this whole time! Would you care to elaborate on "The plugins will run asynchronously, but not in parallel" ?

Sergey.quixoticaxis.Ivanov Over a year ago

You can output CurrentThread's id and you will see that plugins at least start on a single thread. Here's a nice article about it: blogs.msdn.microsoft.com/benwilli/2015/09/10/…

pere57 Over a year ago

@Sergey.quixoticaxis.Ivanov For running in parallel, might it not be better to use AsParallel() (as in plugins.AsParallel().Select(p => p.Start())) instead of Task.Run? See here.

Sergey.quixoticaxis.Ivanov Over a year ago

@pere57 I doubt it will make any difference. Also I'm not certain that plugins can get any value out of work "chunking" inside parallel query. It should be measured irl, what is better for this case.

Theodor Zoulias Over a year ago

"The plugins will run asynchronously, but not in parallel." -- This is not correct. Assuming that the Start method returns an incomplete task, in other words that it is truly asynchronous, which is a reasonable assumption to make, the plugins will run concurrently. Multiple asynchronous operations will be in-flight at the same time.

|

Frank Fajardo · Accepted Answer · 2018-09-01 08:25:01Z

3

You can do:

var tasks = new List<Task>();
foreach(var plugin in plugins) 
{
   var task = plugin.Start();
   tasks.Add(task);
}
await Task.WhenAll(tasks);

edited Sep 1, 2018 at 8:25

answered Sep 1, 2018 at 8:13

Frank Fajardo

7,4291 gold badge31 silver badges49 bronze badges

5 Comments

Neil Over a year ago

My pet peeve is if(collection.Count > 0) - use if(collection.Any()). You don't care how many items, just 'is there any?'.

Frank Fajardo Over a year ago

@Neil, very good suggestion! I always keep forgetting that Any() is more efficient in this case. :)

Sergey.quixoticaxis.Ivanov Over a year ago

Or you can ommit Any check as WhenAll can work with zero tasks.

Sergey.quixoticaxis.Ivanov Over a year ago

Also you probably meant Task.WhenAll, not Tasks.WhenAll

Frank Fajardo Over a year ago

@Sergey.quixoticaxis.Ivanov, I didn't know you could pass an empty collection to it, so thanks for the suggestion.

Enigmativity · Accepted Answer · 2018-09-01 09:51:26Z

2

You could use Microsoft's Reactive Framework to ensure that this is awaitable, happens asynchronously and in parallel.

await
    plugins
        .ToObservable()
        .SelectMany(plugin => Observable.FromAsync(() => plugin.Start()))
        .ToArray();

answered Sep 1, 2018 at 9:51

Enigmativity

117k12 gold badges101 silver badges184 bronze badges

Comments

Francesco Bonizzi · Accepted Answer · 2018-09-01 08:29:10Z

0

As you can see Start method returns a Task. I would define a list of plugin loading tasks and check with Task.WhenAll when every task is completed. After that you can assume all Start methods have returned.

List<IPlugin> plugins = ... 
var pluginsLoadingTasks = new List<Task>();

foreach(var plugin in plugins)
{
    pluginsLoadingTasks.Add(plugin.Start());
}

// It's not necessary to check if pluginsLoadingTasks is empty, 
// because WhenAll won't throw an exception in that case
await Task.WhenAll(pluginsLoadingTasks);
// You can assume all Start methods have completed

I suggest you to read the differences between Task.WhenAll and Parallel.ForEach constructs.

answered Sep 1, 2018 at 8:29

Francesco Bonizzi

5,3646 gold badges57 silver badges91 bronze badges

1 Comment

Dmytro Mukalov Over a year ago

How is your answer different from what Frank suggested?

Collectives™ on Stack Overflow

Awaiting tasks from multiple objects

4 Answers 4

8 Comments

5 Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

8 Comments

5 Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related