Google StackDriver correlating logs with parent request python 3

Question

In python 2.7, the app engine sdk was doing the work in the background to nest all logs with the parent request to have a correlation in Google StackDriver.

As of the transition to python 3, it is through the usage of google cloud logging or structured logging, and from all the different references I could found, it's important to have the same trace id in the 'sub' logs for stack driver to make a match with the 'request' log.

And still as you can see below, it still appear as different logs.

For context, I even tried this on an empty django project deployed on app engine.

Got the same result, even when following the example in the documentation: https://cloud.google.com/run/docs/logging#writing_structured_logs

Trying to log to the stdout is giving the same result.

Edit:

After the initial request, all other request will be nested under the initial request when using the stdout.

But, the highest severity of the 'child' logs is not taken by the 'parent' log, therefore the filters won't pick up the actual log. See below:

Dina Graves Portman · Accepted Answer · 2019-11-15 23:52:44Z

1

Thanks for the question!

It looks like you're logging the trace correctly, but your logName indicates that you're not using a stdout or stderr. If you use one of these for your logs, they will correlate properly, like this:

StackDriver Logs Screenshot

You can see that the logName ends with stdout. An stdout or stderr will correlate. You can create this as shown here in the tutorial:

# Build structured log messages as an object.
global_log_fields = {}

# Add log correlation to nest all log messages
# beneath request log in Log Viewer.
trace_header = request.headers.get('X-Cloud-Trace-Context')

if trace_header and PROJECT:
    trace = trace_header.split('/')
    global_log_fields['logging.googleapis.com/trace'] = (
        f"projects/{PROJECT}/traces/{trace[0]}")

# Complete a structured log entry.
entry = dict(severity='NOTICE',
             message='This is the default display field.',
             # Log viewer accesses 'component' as jsonPayload.component'.
             component='arbitrary-property',
             **global_log_fields)

print(json.dumps(entry))

EDIT:

To filter out the stdout and only see the Request logs in the stackdriver UI, you can de-select the stdout from the filter. Logs Filter

For a sample using the python client API, please see this article and attached sample Flask app. Combining correlated Log Lines in Google Stackdriver

edited Nov 15, 2019 at 23:52

answered Nov 15, 2019 at 0:22

Dina Graves Portman

1515 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

PCS-I Over a year ago

Thanks! Unfortunately, trying it as I said is not correlating the logs for me. I did try it with a stdout and the result was the same, see the updated screenshot.

Dina Graves Portman Over a year ago

Can you try one more time? When you re-deploy the app, this message will be correlated with the log: "This request caused a new process to be started for your application, and thus caused your application code to be loaded for the first time. This request may thus take longer and use more CPU than a typical request for your application." -- I see the same thing when I re-deploy my app. But when I reload the page, the custom message shows correlated with the request log.

PCS-I Over a year ago

You're right! It did appear inside the request! But the log appears twice: once inside the request and once in its own line. Would you know how to make the extra one disappear?

PCS-I Over a year ago

Another issue is that the 'child' log severity is not taken by the 'parent' log (request), which makes all request log with a default severity.

Dina Graves Portman Over a year ago

Added to the answer: You can easily filter out the stdout and only see the Request logs using the filter in the UI.

|

Maulik Patel · Accepted Answer · 2020-05-18 13:17:48Z

0

I was able to achieve this kind of logging structure on Google Cloud Logging Console:

I was using the Django Framework. I wrote Django middleware which integrates Google Cloud Logging API.

"Trace" needs to be added to every log object which points to its parent log object.

please check manage nesting of logs in Google Stackdriver with Django

Please check django-google-stackdriver-nested-logging log_middleware.py source on Github.

answered May 18, 2020 at 13:17

Maulik Patel

417 bronze badges

Collectives™ on Stack Overflow

Google StackDriver correlating logs with parent request python 3

2 Answers 2

8 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

8 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related