How to abstract DB connection in AWS Lambda?

Question

I'm building an application in AWS with quite a few Lambda functions to write - and all of them will create a database instance to query on, by running the following code:

mydb = mysql.connector.connect(
host="endpoint.rds.amazonaws.com",
user="user",
passwd="password",
database="dbname"
)

Now, I don't want to have to explicitly include this code in every single Lambda function - I'd rather put it somewhere else (either in a Layer or a separate Lambda function), so that this could be done simply by something like this:

mydb = ConnectToDB()

Any thoughts on how to do this?

if you are willing to use AWS aurora, It supports IAM authentication to the db now. docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/… — Arun Kamalanathan
– Arun Kamalanathan, Commented Dec 12, 2019 at 21:00

Ethan Harris · Accepted Answer · 2019-12-12 21:12:02Z

2

You have the right idea, assuming you are using python i would create a layer package similar to the following:

python/
    myPackage.py
    mysql/

Where mysql includes the mysql package and myPackage.py includes some variation of:

import mysql

def ConnectToDB(**kwargs):
      return mysql.connector.connect(
                host=kwargs.get("YOUR_ENDPOINT"),
                user=kwargs.get("YOUR_USER"),
                passwd=kwargs.get("YOUR_PASSWORD"),
                database=kwargs.get("YOUR_DBNAME")
          )

Then use this script to create a layer in lambda:

#!/bin/bash

#Required variables
LAYER_NAME="YOUR_LAYER_NAME"
LAYER_DESCRIPTION="YOUR_LAYER_DESCRIPTION"
LAYER_RUNTIMES="python3.6 python3.7"
S3_BUCKET="YOUR_S3_BUCKET"

#Zip Package Files
zip -r ${LAYER_NAME}.zip .
echo "Zipped ${LAYER_NAME}"

#Upload Package to Lambda
aws s3 cp ./${LAYER_NAME}.zip s3://${S3_BUCKET}

#Create new layer
aws lambda publish-layer-version --layer-name ${LAYER_NAME} --description "${LAYER_DESCRIPTION}" --content S3Bucket=${S3_BUCKET},S3Key=${LAYER_NAME}.zip --compatible-runtimes ${LAYER_RUNTIMES}

#Cleanup zip files
rm ${LAYER_NAME}.zip

you can then associate the layer with your lambda function, and import your package in the lambda using the following syntax:

from myPackage import ConnectToDB

connectionParams = {
    "YOUR_ENDPOINT" : ...,
    "YOUR_USER": ...,
    "YOUR_PASSWORD": ...,
    "YOUR_DBNAME": ...
}
mydb = ConnectToDB(**connectionParams)

edited Dec 12, 2019 at 21:12

answered Dec 12, 2019 at 20:58

Ethan Harris

1,36210 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Manuel T. Coelho Over a year ago

Thanks - I did that and it was pretty easy. I'm going to create a config file with the credentials and endpoints and then fetch them form that file.

Ethan Harris Over a year ago

Nice! the only thing i would say is since you are already using AWS lambda, i would use the Secrets Manager service to house your connection strings and API keys rather than a config file stored somewhere

Manuel T. Coelho · Accepted Answer · 2019-12-12 20:57:40Z

2

Solved! I created a python file called DBConnections.py, which includes the function below - and I included it in the deployment package for my AWS Lambda Layer.

def Connect():

mydb = mysql.connector.connect(
host="endpoint.amazonaws.com",
user="user",
passwd="password",
database="mydbname"
)

return mydb

After it was deployed, the only thing I have to do to invoke this is:

from DBConnections import Connect
mydb = Connect()

Voilà.

answered Dec 12, 2019 at 20:57

Manuel T. Coelho

535 bronze badges

Collectives™ on Stack Overflow

How to abstract DB connection in AWS Lambda?

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related