How to avoid using using BuildServiceProvider method at multiple places?

Calling BuildServiceProvider multiple times can cause serious trouble, because each call to BuildServiceProvider results in a new container instance with its own cache. This means that registrations that are expected to have the Singleton lifestyle, suddenly are created more than once. This is a problem called Ambiguous Lifestyle.

Some Singletons are stateless and for them there is no difference in creating one or a thousand. But other components that are registered as Singleton might have state, and the working of the application might (indirectly) depend on that state not being duplicated.

To make matters worse, while your application might work correctly today, this might change any time in the future when one of the third-party or framework components you depend on makes a change to one of their components in such way that it becomes a problem when that component is created multiple times.

In your example, you are resolving both ILoggingService and IProcessClient from a service provider. If the resolved components are stateless objects without stateful dependencies, there is no real harm done. But this might change when they become stateful. Again, this can happen by a change of one of its indirect dependencies, so this is something you might not be aware of. This can cause you or your team to waste many hours; such problem will likely not be easily found.

This means that the answer "simply" is to prevent calling BuildServiceProvider() to create intermediate container instances. But this might be easier said than done. In your case, specifically, you seem to require a dependency on ILoggerService before all dependencies are wired. A typical way to achieve this is to split the registration phase into two separate steps:

• One step where you manually create those few singletons that you need before hand
• Add them to your container builder (IServiceCollection)
• Add all other registrations

For instance:

private ILoggingService _loggingService;

public Startup(Confiration config)
{
    _loggingService = new MySpecialLoggingService(config.LogPath);
}

public void ConfigureServices(IServiceCollection services)
{
    services.AddSingleton(_loggingService);

    // More stuf here.
    ...
}

Advantage of this structure is that when a dependency is added to the constructor of this manually built MySpecialLoggingService, your code stops compiling and you're forced to look at this code. When that constructor depends on some other framework abstraction or application abstraction that isn't yet available, you know you're in trouble and need to rethink your design.

Final note, calling BuildServiceProvider multiple times isn't a bad thing per se. It is okay when you explicitly want to have multiple isolated modules in your application that each have their own state and run independently of one another. For instance when running multiple end points for multiple bounded contexts within the same process.

UPDATE

I think I am starting to understand what it is you are trying to achieve in your BaseMiddleware. It is a 'convenient' helper class that holds all the dependencies that its derivatives might need. This is probably an old design and you might be aware of this, but this base class is quite problematic. Base classes with dependencies are hardly ever a good idea, because they tend to become big, ever changing, and obfuscate the fact that their derivatives become too complex. In your case, even, you are using the Service Locator anti-pattern which is never a good idea.

Besides this, there is a lot going on in that BaseMiddleware class that—to me—makes little sense, such as:

• It contains complex logic to verify whether all dependencies exist, while there are more effective ways to do so. The most effective way is to apply Constructor Injection because it will guarantee that its necessary dependencies are always available. On top of that, you can validate the IServiceCollection on build. This gives you even greater guarantees over the correctness of your DI configuration than your BaseMiddleware currently provides.
• It resolves all its services in background threads, which implies that construction of those components is either heavy on CPU or I/O, which is a problem. Instead composition should be fast, because injection constructors should be simple, which allows you to compose object graph with confidence.
• You do exception handling in the base class, while it is better suited to be applied at a higher level; for instance, using an outer-most piece of middleware. For the sake of simplicity, though, my next example keeps exception handling inside the base class. That's because I have no idea what kind of things you are doing in there, that could influence my answer.
• As the base class is resolving from the root container, middleware classes are only able to make use of Singleton dependencies. Connecting to the database through Entity Framework, for instance, will be a problem, because DbContext classes should not be captured in Singleton consumers.

So, with the observations and advice above, I would suggest reducing the BaseMiddleware class to the following:

// Your middleware classes should implement IMiddleware; this allows middleware
// classes to be transient and have scoped dependencies.
public abstract class ImprovedBaseMiddleware : IMiddleware
{
    public abstract bool IsCorrectEndpoint(HttpContext context);
    public abstract string GetEndpoint(HttpContext context);
    public abstract Task HandleRequest(HttpContext context);

    public async Task InvokeAsync(HttpContext context, RequestDelegate next)
    {
        if (IsCorrectEndpoint(context))
        {
            try
            {
                await HandleRequest(context);
            }
            catch (Exception ex)
            {
                // handle exception here
                return;
            }
            return;
        }

        await next(context);      
    }
}

Now based on this new base class, create your middleware implementations similar to this next example:

public class ImprovedFirstServiceMiddleware : ImprovedBaseMiddleware
{
    private readonly ICatalogService _catalogService;
    
    // Add all dependencies required by this middleware in the constructor.
    public FirstServiceMiddleware(ICatalogService catalogService)
    {
        _catalogService = catalogService;
    }

    public override bool IsCorrectEndpoint(HttpContext context) =>
         context.Request.Path.StartsWithSegments("/first");

    public override string GetEndpoint(HttpContext context) => "/first";

    public override async Task HandleRequest(HttpContext context)
    {
        context.Response.StatusCode = 200;
        context.Response.ContentType = "application/json";
        await context.Response.WriteAsync("Hello from "
            + _catalogService.SomeValue());
    }
}

In your application, you can register your middleware classes as follows:

public void ConfigureServices(IServiceCollection services) {
    // When middleware implements IMiddleware, it must be registered. But
    // that's okay, because it allows the middleware with its
    // dependencies to be 'verified on build'.
    services.AddTransient<ImprovedFirstServiceMiddleware>();
   
    ...
}

public void Configure(
    IApplicationBuilder app, IHostApplicationLifetime lifetime)
{
    // Add your middleware in the correct order as you did previously.
    builder.UseMiddleware<ImprovedFirstServiceMiddleware>();
}

TIP: If you start to notice that a middleware classes get big constructors, that's likely because such class does too much and gets too complex. This means it should be refactored into multiple smaller classes. In that case, your class is exhibiting the Constructor Over-Injection code smell. There are many possible refactoring patterns and design patterns available that can get you out of this situation.