I'm trying to figure out how to add middleware to a chat client added to the Kernel. It seems like I need the actual instance of the chat client in order to add the middleware?
This is the documentation for the chat client middleware https://learn.microsoft.com/en-us/dotnet/ai/microsoft-extensions-ai#custom-ichatclient-middleware
This is my setup
services.AddKeyedTransient<Kernel>("Kernel", (sp, _) =>
{
...
var bedrockRuntime = new AmazonBedrockRuntimeClient(awsCredentials, clientConfig);
var kernelBuilder = Kernel.CreateBuilder();
kernelBuilder.AddBedrockChatClient(
modelId: awsBedrockOptions.GetModel(LlmOptions.ModelPurpose.General),
bedrockRuntime: bedrockRuntime
);
return kernelBuilder.Build();
});
This is the middleware:
public sealed class LlmLoggerChatClient(
IChatClient innerClient,
LlmLogger llmLogger)
: DelegatingChatClient(innerClient)
{
public override async Task<ChatResponse> GetResponseAsync(
IEnumerable<ChatMessage> messages,
ChatOptions? options = null,
CancellationToken cancellationToken = default)
{
var response = await base.GetResponseAsync(messages, options, cancellationToken);
await llmLogger.LogAsync(...)
return response;
}
public override async IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
IEnumerable<ChatMessage> messages,
ChatOptions? options = null,
[EnumeratorCancellation] CancellationToken cancellationToken = default)
{
await foreach (var update in base.GetStreamingResponseAsync(messages, options, cancellationToken))
{
yield return update;
}
}
}
And this is how I finally use the kernel
ChatCompletionAgent agent = new()
{
Name = "Search Agent",
Kernel = kernel, // Injected via DI
...
};
var response = await agent.InvokeAsync(...
At what point should I/can I extend the Bedrock chat client?