Parsing SNS event Containing S3 events in Lambda

Question

I required multiple lambdas to subscribe to s3 bucket events, this is not supported so the idea is to introduce the SNS instead.

The flow will be S3Event -> SNS -> (X Number of LAMBDA consumers)

The setup and architecture is find, and the Lambda is being triggered correctly with the Event type

SNSEvent with the `ILambdaContext. This is fine.

My problem is when trying to parse the S3 events from the SNSEvent Message.

foreach (var record in snsEvent.Records)
{
    var snsRecord = record.Sns;
    if (string.IsNullOrEmpty(snsRecord.Message))
    {
      logger.Error("Message is empty", record);
      throw new Exception("Message is empty");
    }

  // Object reference not set to an instance of an object.
  var s3Event = JsonSerializer.Deserialize<S3Event>(snsRecord.Message);
  logger.Info($"{s3Event.Records.FirstOrDefault()?.S3.Object.Key}", snsEvent);

  await ProcessParsedEvent(s3Event, context);
}

The exception is not throw for an empty snsRecord.Message however the cloudwatch is a generic object reference error. However I can't seem to see it.

I suspected there were more than 1 event of a different type which maybe be causing issues with the deserializer, so i added :

var records = snsEvent.Records.Where(x => x.EventSource.ToLowerInvariant() == "aws:s3");

However the same results.. Is there something specific I need to do for deserializing the s3 events ?

Ermiya Eskandary · Accepted Answer · 2023-12-31 19:29:08Z

0

The problem is related to the Deserialiazer's property case sensitivity.

The solution in my case was to add the PropertyNameCaseInsensitive in the JsonSerializerOptions set to true.

From here the SNS message (which contains the s3 event message in JSON format, lower case, is able to deserialize correctly.

  var deserializeSettings = new JsonSerializerOptions
  {
      PropertyNameCaseInsensitive = true
  };

  var s3 = JsonSerializer.Deserialize<S3Event>(s3Json, deserializeSettings);

edited Dec 31, 2023 at 19:29

Ermiya Eskandary

24.4k4 gold badges58 silver badges72 bronze badges

answered Dec 30, 2023 at 22:01

Pogrindis

8,3365 gold badges34 silver badges44 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Parsing SNS event Containing S3 events in Lambda

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related