0

I am trying to integrate application logs running on EC2 instances with Grafana Loki using FluentD & Fluent-bit.

I am facing the following:

  1. Fluent-bit service is running normally
  2. The application logs don't shipping
  3. I see this info/warning from monitoring fluent-bit.service:

[debug] [input:tail:tail.3] scan_blog add(): dismissed: /opt/SP/users/oracle/ocomc/logs/csp-nmgr-1/ocomc_IoT.log, inode 100987803

[debug] [input:tail:tail.3] scan_blog add(): dismissed: /opt/SP/users/oracle/ocomc/logs/event-nmgr-1/ocomc_IoT.log, inode 100727404

[debug] [input:tail:tail.3] scan_blog add(): dismissed: /opt/SP/users/oracle/ocomc/logs/session-nmgr-1/ocomc_IoT.log, inode 854909

[debug] [input:tail:tail.3] 0 new files found on path '/opt/SP/users/oracle/ocomc/logs/-nmgr/ocomc_IoT.log'

[debug] [input:tail:tail.3] scanning path /opt/SP/users/oracle/ocomc/logs/-nmgr/ocomc_IoT.log

[debug] [input:tail:tail.3] scan_blog add(): dismissed: /opt/SP/users/oracle/ocomc/logs/collection-nmgr-1/ocomc_IoT.log, inode 84468627

[debug] [input:tail:tail.3] scan_blog add(): dismissed: /opt/SP/users/oracle/ocomc/logs/csp-nmgr-1/ocomc_IoT.log, inode 100987803

[debug] [input:tail:tail.3] scan_blog add(): dismissed: /opt/SP/users/oracle/ocomc/logs/event-nmgr-1/ocomc_IoT.log, inode 100727404

[debug] [input:tail:tail.3] scan_blog add(): dismissed: /opt/SP/users/oracle/ocomc/logs/session-nmgr-1/ocomc_IoT.log, inode 854909

[debug] [input:tail:tail.3] 0 new files found on path '/opt/SP/users/oracle/ocomc/logs/-nmgr/ocomc_IoT.log'

[debug] [input:tail:tail.3] scanning path /opt/SP/users/oracle/ocomc/logs/-nmgr/ocomc_IoT.log

[debug] [input:tail:tail.3] scan_blog add(): dismissed: /opt/SP/users/oracle/ocomc/logs/collection-nmgr-1/ocomc_IoT.log, inode 84468627

[debug] [input:tail:tail.3] scan_blog add(): dismissed: /opt/SP/users/oracle/ocomc/logs/csp-nmgr-1/ocomc_IoT.log, inode 100987803

[debug] [input:tail:tail.3] scan_blog add(): dismissed: /opt/SP/users/oracle/ocomc/logs/event-nmgr-1/ocomc_IoT.log, inode 100727404

[debug] [input:tail:tail.3] scan_blog add(): dismissed: /opt/SP/users/oracle/ocomc/logs/session-nmgr-1/ocomc_IoT.log, inode 854909

[debug] [input:tail:tail.3] 0 new files found on path '/opt/SP/users/oracle/ocomc/logs/-nmgr/ocomc_IoT.log'

Please advise.

Thanks

Improve this question

1 Answer 1

Reset to default
0

The root cause for that issue was another heavy system that was using the same Grafana configurations and putting a heavy load on the monitoring system, which resulted in blocking other systems from accessing Grafana. Once we stopped that system, everything returned to normal.

I wanted to share this information with you in case someone else faces the same issue. The root cause was not initially apparent in our troubleshooting, but we eventually discovered it.

Thanks

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.