How to update log stream name in CloudWatch in AWS using fluent bit

Question

I am interested in pod logs. I am trying to ship logs from AWS EKS to cloudwatch using below fluent-bit deamon set in EKS

https://raw.githubusercontent.com/aws-samples/amazon-cloudwatch-container-insights/latest/k8s-deployment-manifest-templates/deployment-mode/daemonset/container-insights-monitoring/fluent-bit/fluent-bit.yaml

I am applying it, but it creates the log stream in format. How can I just have instance name and pod name in it? Can someone help?

The current format is

ip-10-11-97-64.ec2.internal-application.var.log.containers.sonarqube-sonarqube-dce-search-0_sonarqube-dce_sonarqube-dce-search-b892adb2fbf60a36568c3578c20181941dc8f6d00a8180cb837f9f76ecaa515a.log

I am interested to have log stream name as instancename_namespacename_podname

Saifeddine Rajhi · Accepted Answer · 2024-09-22 14:20:43Z

1

This is totally possible using the variables log_stream_name or if you want some regex log_stream_prefix in ConfigMap resource before applying it.

You can download the yaml from the Github raw link, then edit it by changing in the ConfigMap:

log_stream_prefix   ${HOST_NAME}.

to something like:

log_stream_prefix    /eks/$(kubernetes['namespace_name'])/$(kubernetes['pod_name'])

answered Sep 22, 2024 at 14:20

Saifeddine Rajhi

2221 silver badge12 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to update log stream name in CloudWatch in AWS using fluent bit

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related