0

Ok. So I am going to start by apologizing up front as I know there have been a lot of question asked about Parallel and Async. However, even after searching I can not wrap my brain around how this should work. Coding is something I dabble in not something I do day in and day out.

I am trying to help a friend collect some historic stock data using the AlphaVantage API.

I have no issues collecting the data and saving it to the database but it can take a long time to pull 20 years of daily prices. So to avoid the GUI freezing up I made it an async function. However, Considering how often and how much data needs to be pulled it will need to be done in parallel. I am not sure how many parallel events yet but the long term goal is to have a timer in the application that fires off at intervals and calls the BatchUpdateStockHistoryAsync method and passes in a value for how many stocks to pull and update in this batch the function will then go out, query the DB and get a list 5 that have the oldest update.

For the moment I stripped all of that out to simply the question and just created a manual list of 5 stocks that get iterated through. For each stock in the list it calls another function PullAndWriteHistoricDailyData(item) that does all the work of actually reaching out to AlphaVantage and updating the DB.

After this lengthy info my question is what's the best way is to trigger multiple concurrent threads of PullAndWriteHistoricDailyData()?

I imagine I will need to also play with MaxDegreeOfParallelism to figure out what works best but I have not even gotten that far yet. I believe I would want to use Parallel.ForEachAsync maybe? but just not sure how to pull it all together.

Any help would be greatly appreciated

Misiu

    public async Task BatchUpdateStockHistoryAsync()
    {
        List<string> list = new List<string>();

        list.Add("AAPL");
        list.Add("F");
        list.Add("MSFT");
        list.Add("COKE");
        list.Add("IAG");

        foreach(string item in list)
        {
            await Task.Run(() => PullAndWriteHistoricDailyData(item));
        }

        return completed;
    }
Improve this question
4
  • For an example of using the Parallel.ForEachAsync method, you could look here. Commented Apr 8, 2022 at 17:22
  • 1
    @ThomasWeller Not at all and I spent a few hours reading up on various things before I posted. But I clearly have a learning deficiency when it comes to this because I was not able to connect the dots. Commented Apr 8, 2022 at 17:33
  • Since you said PullAndWriteHistoricDailyData is calling an external web API (AlphaVantage), you should rewrite that method to be async. For example, it could use HttpClient.GetAsync to make the call to AlphaVantage. Just wrapping a synchronous method in a Task.Run isn't going to magically make it asynchronous. Commented Apr 8, 2022 at 17:44
  • 1
    @MattJohnson-Pint Thank you Matt. I will give that a shot as well. Its actually using GetStringFromURL() but there is a GetStringFromURLAsync() I will try next. Commented Apr 8, 2022 at 18:07

1 Answer 1

Reset to default
1

EDIT I mistakenly read about Task.WhenAll vs Parallel.ForEach originally. After reading more about Parallel.ForEachAsync based on the answer by Theodore Zoulias at: is-parallel-foreachasync-a-replacement-to-a-plain-for-loop-append-to-task-list, and based on the assumption that you want to view the result of each Task as apposed to receiving a single Task from the call to Parallel.ForEachAsync, then I would propose changing to using the following:

You could switch to using a List<Task> and use await Task.WhenAll() and skip the Parallel.ForEachAsync like so:

 List<Task> myupdatetasks = new List<Task>();

 foreach(string item in list)
 {
     myupdatetasks.Add(Task.Run(() => PullAndWriteHistoricDailyData(item)));
 }

 await Task.WhenAll(myupdatetasks).ConfigureAwait(false);

if PullAndWriteHistoricDailyData() is already an async method, you don't need to use Task.Run, you could do the same as above minus the Task.Run

 List<Task> myupdatetasks = new List<Task>();

 foreach(string item in list)
 {
     myupdatetasks.Add(PullAndWriteHistoricDailyData(item));
 }

 await Task.WhenAll(myupdatetasks).ConfigureAwait(false);

if you are intent on using Parallel.ForEachAsync, this article explains it pretty well: parallelforeachasync-in-net-6

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

Could you name the reasons that the Task.WhenAll is preferable to Parallel.ForEachAsync?
Thank you Ryan I appreciate it. PullAndWriteHistoricDailyData() is not currently an async method. I will give this a shot now and then mark solved. No idea why I had such a hard time coming to that conclusion on my own.
@TheodorZoulias Although, I'm going to assume you may refer me to your answer here: /is-parallel-foreachasync-a-replacement-to-a-plain-for-loop-append-to-task-list :) After reading your answer, I would wager that using this as apposed to Parallel.ForEachAsync would be desirable if you wanted to inspect the result of each Task instead of getting back a "naked" Task with Parallel.ForEachAsync
Ryan the OP has explicitly stated that they consider using the Parallel.ForEachAsync API, both in the title and the body of the question. So I assumed that you might have some good reasons for suggesting them to skip it, and switch to the Task.WhenAll API instead. If the reasons are just the lack of familiarity with .NET 6, you should probably edit the answer clarify this point.
@TheodorZoulias I believe I have familiarized myself enough with Parallel.ForEachAsync based on your answer I linked to above, and I gave you a reason why this would be a desirable solution based on the hypothetical that the developer wants to see the result of each Task as apposed to getting back what you referred to as a "naked" task via the call to Parallel.ForEachAsync, if that isn't a good enough reason, then I guess we can update the answer to add detail that this is what you could do using .Net 5 and below. Or I'll update my answer to include this as a reason for using this.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.