I have a data layer that is created and updated from multiple input layers. We currently use functions and a Service Bus session to ensure that only a single worker is ever trying to update a data layer at a time, even if there are multiple input layers that need to be aggregated in the queue. We'd like to use a Durable Orchestration though, since the aggregation function is long running with multiple steps. Is there a way to ensure that a single orchestration is only ever running for a Data Layer ID, and the orchestration is still invoked for all pending Input Layers?
I've used an InstanceId, but calling StartNew with an instance ID that is already running errors (I was hoping it would queue).
I've thought of creating eternal orchestrators that loop and wait for external events, since external events queue. My understanding though is this would create an eternal orchestrator per Data Layer ID, which are significant.
I've also thought of a still using Service Bus sessions, and have the function that invokes the orchestrator and monitor its progress - but this would then keep a long running function active that is, technically, doing nothing - not sure how that would impact the runtime (would be pretty light weight in theory, just a sleep/check loop)?
I've also looked at durable entities to trigger the orchestrator, but I don't think there is a reliable way to block within the durable entity method and wait for the orchestration to complete - and it may be beyond the intended use case of the durable entity anyway since we wouldn't use it to store state, just for coordination.
