How do I read real time AWS Lambda logs using on-prem Python script?

Question

I do not want to wait for 14 seconds to read CloudWatch Lambda logs. Is there any way to read them in realtime in invoking shell on premises (outside VPC)?

Invoke async

    result = lam.invoke(
        InvocationType='Event',
        FunctionName='my-lambda-func',
        Payload=json.dumps(dict(test='test'))
    )

Lambda writes logs to a service in question.
Loop/wait for real-time results in the same on separate shell.

Why not simply return the data rather than trying to extract it from logs? — John Rotenstein
– John Rotenstein, Commented Aug 23, 2021 at 23:26
@JohnRotenstein Async invocation does not return output from lambda in that case. — Marcin
– Marcin, Commented Aug 24, 2021 at 0:21
Then call it synchronously instead of asynchronously and simply return the data. Why does the OP want to read the data from logs? — John Rotenstein
– John Rotenstein, Commented Aug 24, 2021 at 1:43

Marcin · Accepted Answer · 2021-08-24 00:25:41Z

1

A common way to use async invocation is to have your lambda publish your results to SQS queue or SNS topic. This way you can pull SQS queue on premise for results, or setup http endpoint subscription which will be automatically invoked by SNS when it gets a message from the lambda.

If you only want to focus on CloudWatch logs, then you can setup subscription filter on the logs for real-time processing of the incoming log events from lambda. Depending on your exact setup, you can use Kinesis, firehose or other lambda to get the log messages.

answered Aug 24, 2021 at 0:25

Marcin

241k16 gold badges315 silver badges368 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How do I read real time AWS Lambda logs using on-prem Python script?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related