0

I have a tomcat installation which has a few Java Spring Boot application deployed. This causes two different log pattern in the catalina.out logfile: Default tomcat logging:

18-Nov-2024 14:15:16.898 INFO [Catalina-utility-2] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/opt/tomcat/webapps/connector#api.war] has finished in [3,731] ms

and Spring Boot logging:

2024-12-03T09:04:40.762Z  WARN 865 --- [o-8443-exec-247] .w.s.m.s.DefaultHandlerExceptionResolver : Resolved [org.springframework.web.HttpRequestMethodNotSupportedException: Request method 'GET' is not supported]

Because Boot Spring sometimes has an empty line between the beginning a log message and the corresponding stacktrace the multiline parser "java" does not work. Thats why I have integrated the following configuration:

fluent-bit.conf:

[SERVICE]
    flush        1
    daemon       Off
    log_level    debug
    parsers_file parsers.conf
    http_server  Off
    http_listen  0.0.0.0
    http_port    2020
    storage.metrics on

[FILTER]
    Name record_modifier
    Match *
    Record hostname ${HOSTNAME}

[INPUT]
    name tail
    path /tmp/catalina.out
    Tag tomcatLog
    multiline.parser multiline-tomcat-default, multiline-tomcat-springboot

[FILTER]
    Name Parser
    Match tomcatLog
    Key_name log
    parser tomcat-springboot
    parser tomcat-default

[OUTPUT]
    name  stdout
    match *

parsers.conf:

[PARSER]
    Name tomcat-default
    Format regex
    Regex ^(?<time>(\d{2}-\w*-\d{4}\s[\d|:|.]*))\s(?<level>\w*)\s(?<message>.*(?:\r?\n.*)*)
    Time_Key time
    Time_Format %d-%b-%Y  %H:%M:%S.%L

[MULTILINE_PARSER]
    Name multiline-tomcat-default
    Type regex
    flush_timeout 1000
    rule "start_state" "/^(?<time>(\d{2}-\w*-\d{4}\s[\d|:|.]*))\s(?<level>\w*)\s(?<message>.*)$/" "cont"
    rule "cont" "/(^\s*$)|(^\s.*$)/" "cont"

[PARSER]
    Name tomcat-springboot
    Format regex
    Regex ^(?<time>(\d|-|:|\.|T|Z)*)\s*(?<level>\w*?)\s(?<pid>\d*)\s-{3}\s\[(?<threadName>.*)\]\s(?<class>.*)\s:\s(?<message>.*(?:\r?\n.*)*)
    Time_Key time
    Time_Format %Y-%m-%dT%H:%M:%S.%LZ

[MULTILINE_PARSER]
    Name multiline-tomcat-springboot
    Type regex
    flush_timeout 1000
    rule "start_state" "/^(?<time>(\d|-|:|\.|T|Z)*)\s*(?<level>\w*?)\s(?<pid>\d*)\s-{3}\s\[(?<threadName>.*)\]\s(?<class>.*)\s:\s(?<message>.*)/" "cont"
    rule "cont" "/^\D.*$/" "cont"

The default messages get parsed successfully and the temporary key "log" gets removed. When parsing a spring message I can see one successfully parsed message and one with the unparsed message:

[0] tomcatLog: [[1733216505.546000000, {}], {"level"=>"INFO", "pid"=>"865", "threadName"=>"o-8443-exec-234", "class"=>"o.s.web.servlet.DispatcherServlet       ", "message"=>"Completed initialization in 2 ms
"}]
[1] tomcatLog: [[1733216680.757000000, {}], {"level"=>"WARN", "pid"=>"865", "threadName"=>"o-8443-exec-247", "class"=>".m.m.a.ExceptionHandlerExceptionResolver", "message"=>"Failure in @ExceptionHandler com.falke.rdmconnector.rest.GlobalExceptionHandler#handleRuntimeException(RuntimeException)
java.lang.IllegalStateException: Could not resolve parameter [0] in org.springframework.web.ErrorResponse com.falke.rdmconnector.rest.GlobalExceptionHandler.handleRuntimeException(java.lang.RuntimeException): No suitable resolver
        at org.springframework.web.method.support.InvocableHandlerMethod.getMethodArgumentValues(InvocableHandlerMethod.java:221) ~[spring-web-6.1.12.jar:6.1.12]
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:178) ~[spring-web-6.1.12.jar:6.1.12]
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118) ~[spring-webmvc-6.1.12.jar:6.1.12]
        at org.springframework.web.servlet.mvc.method.annotation.ExceptionHandlerExceptionResolver.doResolveHandlerMethodException(ExceptionHandlerExceptionResolver.java:432) ~[spring-webmvc-6.1.12.jar:6.1.12]        at org.springframework.web.servlet.handler.AbstractHandlerMethodExceptionResolver.doResolveException(AbstractHandlerMethodExceptionResolver.java:74) ~[spring-webmvc-6.1.12.jar:6.1.12]
"}]
[2] tomcatLog: [[1733395924.011246623, {}], {"log"=>"        at org.springframework.web.method.support.InvocableHandlerMethod.getMethodArgumentValues(InvocableHandlerMethod.java:221) ~[spring-web-6.1.12.jar:6.1.12]
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:178) ~[spring-web-6.1.12.jar:6.1.12]
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118) ~[spring-webmvc-6.1.12.jar:6.1.12]
        at org.springframework.web.servlet.mvc.method.annotation.ExceptionHandlerExceptionResolver.doResolveHandlerMethodException(ExceptionHandlerExceptionResolver.java:432) ~[spring-webmvc-6.1.12.jar:6.1.12]        at org.springframework.web.servlet.handler.AbstractHandlerMethodExceptionResolver.doResolveException(AbstractHandlerMethodExceptionResolver.java:74) ~[spring-webmvc-6.1.12.jar:6.1.12]

", "hostname"=>"appdev"}]
[3] tomcatLog: [[1733216680.762000000, {}], {"level"=>"WARN", "pid"=>"865", "threadName"=>"o-8443-exec-247", "class"=>".w.s.m.s.DefaultHandlerExceptionResolver", "message"=>"Resolved [org.springframework.web.HttpRequestMethodNotSupportedException: Request method 'GET' is not supported]"}]

My suspect is that the first (parsed) message is from the first "parser" entry the seconds one is untouched because the second entry fails.

So my question is: How do I parse two different multiline log formats from one source?

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.