15

I'm playing around with the Azure Durable functions. Currently I'm getting InvalidOperationException within Orchestration function after I call an activity. It complains that Multithreaded execution was detected. This can happen if the orchestrator function previously resumed from an unsupported async callback.

Have any one experienced such an issue? What I'm doing wrong? Complete code can be found on GitHub

Here is the line from the orchestration function:

var res = await ctx.CallActivityAsync<int>("LengthCheck", "inputData");

The LengthCheck activitiy function is:

[FunctionName("LengthCheck")]
public static Task<int> Calc([ActivityTrigger] string input)
{
    var task = Task.Delay(TimeSpan.FromSeconds(5));
    task.Wait();
    return Task.FromResult(input.Length);
}

The stack trace is:

ac6fd5cdd07a4dc9b2577657d65c4f27: Function 'InpaintOrchestration (Orchestrator)', version '' failed with an error. Reason: System.InvalidOperationException: Multithreaded execution was detected. This can happen if the orchestrator function previously resumed from an unsupported async callback.

at Microsoft.Azure.WebJobs.DurableOrchestrationContext.ThrowIfInvalidAccess()

at Microsoft.Azure.WebJobs.DurableOrchestrationContext.d__47`1.MoveNext()

End of stack trace from previous location where exception was thrown

at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()

Improve this question

2 Answers 2

Reset to default
14

This exception happens whenever an orchestrator function does async work in an unsupported way. "Unsupported" in this context effectively means that await was used on a non-durable task (and "non-durable" means that it was a task that came from some API other than IDurableOrchestrationContext).

You can find more information on the code constraints for orchestrator functions here: https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-code-constraints.

Here are the rules that were broken in your code when I quickly scanned it:

  • Orchestrator code should be non-blocking. For example, that means no I/O and no calls to Thread.Sleep or equivalent APIs. If an orchestrator needs to delay, it can use the CreateTimer API.

  • Orchestrator code must never initiate any async operation except by using the IDurableOrchestrationContext API. For example, no Task.Run, Task.Delay or HttpClient.SendAsync. The Durable Task Framework executes orchestrator code on a single thread and cannot interact with any other threads that could be scheduled by other async APIs.

This exception specifically occurs when we detect that an unsupported async call is made. I noticed that is happening in this code:

    private static async Task SaveImageLabToBlob(ZsImage imageLab, CloudBlobContainer container, string fileName)
    {
        var argbImage = imageLab
            .Clone()
            .FromLabToRgb()
            .FromRgbToArgb(Area2D.Create(0, 0, imageLab.Width, imageLab.Height));

        using (var bitmap = argbImage.FromArgbToBitmap())
        using (var outputStream = new MemoryStream())
        {
            // modify image
            bitmap.Save(outputStream, ImageFormat.Png);

            // save the result back
            outputStream.Position = 0;
            var resultImageBlob = container.GetBlockBlobReference(fileName);
            await resultImageBlob.UploadFromStreamAsync(outputStream);
        }
    }

The proper way to make async or blocking calls is to wrap them in activity functions, which don't have any of these constraints.

In more recent versions of this extension (v1.3.2 and greater), we've included a link to the documentation describing code-constraints in the exception message.

Improve this answer
Sign up to request clarification or add additional context in comments.

8 Comments

The DurableOrchestrationClient API should be valid as well, shouldn't it? This is useful for getting the Http​Management​Payload from within the orchestrator.
Thanks a lot for this solution. So the way is always -> create activity step and in that method call activity that calls something to get data.
The link doesn't work as @baouss commented, adding updated link: DurableOrchestrationClient Class
I fixed the links in the answer to be up to date (and specifically referenced the V2 APIs). To answer @baouss's question, no, the client APIs are not valid. If you need to use client APIs, do it from an activity function.
Sorry, but what is an activity function though?
|
4

This was happening for my durable orchestrator function too. I had to get rid of all the .ConfigureAwait(false) endings for the activity function invocations.

 //invoking First activity function
        var id = await context.CallActivityAsync<Guid>(Function1, requestModel);
        

        //invoking second activity function that uses data from the first activity function without ConfigureAwait(false)
        var readModel = await context.CallActivityAsync<ReadModel>(Function2, id);
       

        //invoking third activity function that uses data from the second activity function without ConfigureAwait(false)
        await context.CallActivityAsync(Function3, cartReadModel);
Improve this answer

1 Comment

Thank you! I can confirm that as of December 2024 on .NET 8.0 this is still an issue. It appears to be a known constraint that they have just failed to document. github.com/MicrosoftDocs/azure-docs/issues/24080

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.