AWS Lambda JSON to DynamoDB KeyError Serializing JSON

Question

I am setting up my lambda function on AWS and want to insert my JSON into the DynamoDB (NoSQL). I think it has to do with the serialization of the JSON (decimal_serializer), but I might be wrong. Thanks for help!

I get following error while serializing this data:

Error MSG:

{
  "errorMessage": "'male_confidence'",
  "errorType": "KeyError"
}

JSON DATA to INSERT:

{
  "device_id": "abc876",
  "recorded_at": "1496136878",
  "customers": [
    {
      "male_confidence": "0.2",
      "female_confidence": "0.8"
    },
    {
      "male_confidence:": "0.1",
      "female_confidence": "0.9"
    }
  ]
}

Lambda Function Handler

import boto3
import json

def lambda_handler(event, context):
    # TODO implement
    client = boto3.client('dynamodb')
    for customer in event['customers']:
        client.put_item(TableName="cv_data_1", Item={'device_id': {"S": event['device_id']}, 'male_confindence': {"N": customer['male_confidence']}, 'female_confidence': {"N": customer['female_confidence']}, "timestamp":{ "N": event['recorded_at']}})
    print('Successfully processed %s items.' % str(len(event['customers'])))

AWS Output Log Error (Detail):

During handling of the above exception, another exception occurred:

09:23:38
Traceback (most recent call last):

09:23:38
File "/var/runtime/awslambda/bootstrap.py", line 463, in <module>

09:23:38
main()

09:23:38
File "/var/runtime/awslambda/bootstrap.py", line 459, in main

09:23:38
handle_event_request(request_handler, invokeid, event_body, context_objs, invoked_function_arn)

09:23:38
File "/var/runtime/awslambda/bootstrap.py", line 240, in handle_event_request

09:23:38
result = to_json(result)

09:23:38
File "/var/runtime/awslambda/bootstrap.py", line 215, in to_json

09:23:38
return json.dumps(obj, default=decimal_serializer)

09:23:38
File "/var/lang/lib/python3.6/json/__init__.py", line 238, in dumps

09:23:38
**kw).encode(obj)

09:23:38
File "/var/lang/lib/python3.6/json/encoder.py", line 199, in encode

09:23:38
chunks = self.iterencode(o, _one_shot=True)

09:23:38
File "/var/lang/lib/python3.6/json/encoder.py", line 257, in iterencode

09:23:38
return _iterencode(o, 0)

09:23:38
File "/var/runtime/awslambda/bootstrap.py", line 104, in decimal_serializer

09:23:38
raise TypeError(repr(o) + " is not JSON serializable")

09:23:38
TypeError: <FrameSummary file /var/task/lambda_function.py, line 8 in lambda_handler> is not JSON serializable

From the error message, it appears that themale_confidence is missing in one of the customer. (No pun intended). Can you just print the event in the lambda handler so that you can see what data event contains? The output will be in cloudwatch logs. — user1741851
– user1741851, Commented May 30, 2017 at 12:28
{'device_id': 'abc876', 'recorded_at': '1496136878', 'customers': [{'male_confidence': '0.2', 'female_confidence': '0.8'}, {'male_confidence:': '0.1', 'female_confidence': '0.9'}]} , but if I output print(customer)in the loop it returns me the dictsee below. — zer02
– zer02, Commented May 30, 2017 at 12:42
I printed customer['male_confidence'] in the loop and it returns me 0.2, the second loop breaks with the error above. — zer02
– zer02, Commented May 30, 2017 at 12:45
{'device_id': 'abc876', 'recorded_at': '1496136878', 'customers': [{'male_confidence': '0.2', 'female_confidence': '0.8'}, {'male_confidence:': '0.1', 'female_confidence': '0.9'}]} HELLO {'male_confidence': '0.2', 'female_confidence': '0.8'} ##### 0.2 0.8 HELLO {'male_confidence:': '0.1', 'female_confidence': '0.9'} 'male_confidence': KeyError Traceback (most recent call last): File "/var/task/lambda_function.py", line 14, in lambda_handler mc = customer['male_confidence'] KeyError: 'male_confidence' — zer02
– zer02, Commented May 30, 2017 at 12:47

Adam Owczarczyk · Accepted Answer · 2017-05-30 13:04:20Z

1

Typo:

  "male_confidence:": "0.1",
  "female_confidence": "0.9"

"male_confidence:"

answered May 30, 2017 at 13:04

Adam Owczarczyk

2,8621 gold badge19 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

zer02 Over a year ago

THANKS ALOT! I was going crazy that :.

Raf · Accepted Answer · 2017-05-30 13:06:31Z

1

You have a typo after male_confidence:, note the extra :. Try this

{
  "device_id": "abc876",
  "recorded_at": "1496136878",
  "customers": [
    {
      "male_confidence": "0.2",
      "female_confidence": "0.8"
    },
    {
      "male_confidence": "0.1",
      "female_confidence": "0.9"
    }
  ]
}

answered May 30, 2017 at 13:06

Raf

10.2k1 gold badge34 silver badges42 bronze badges

Collectives™ on Stack Overflow

AWS Lambda JSON to DynamoDB KeyError Serializing JSON

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related