How to execute bash commands in aws lambda

Question

I am trying to schedule a job in AWS lambda where i get data fromm a Json API. I want to transfer JSON file to amazon S3 everytime.I have set up S3 bucket and aws lambda function with proper IAM roles. I am writing AWS lambda function in Python. Code works fine on an EC2 instance but It's not transferring file to S3 if I put it in AWS Lambda.

import os


def lambda_handler(event, context):
    #changing the directory to /tmp
    os.chdir("/tmp")
    print "loading function"
    #downloading file to 
    os.system("wget https://jsonplaceholder.typicode.com/posts/1 -P /tmp")
    #using aws-cli to transfer file to amazon S3
    os.system("aws s3 sync . s3://targetbucket")

I am new to aws lambda. I am not getting any error but it's not giving me expected output

franklinsijo · Accepted Answer · 2017-02-22 21:13:26Z

3

AWS Lambda does not have the aws cli by default.

You can either Create a deployment package with awscli in it or Use python boto3 library.

import boto3

s3client = boto3.client('s3')
for filename in os.listdir('/tmp'): # assuming there will not be any sub-directories
    fpath = os.path.join('/tmp',filename)
    if os.path.isfile(fpath): 
        s3client.upload_file(fpath, 'targetbucket', filename)

edited Feb 22, 2017 at 21:13

answered Feb 22, 2017 at 20:33

franklinsijo

18.4k4 gold badges50 silver badges66 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

liferacer Over a year ago

Thanks for your reply. Your code is working fine. It seems I am not able to download file using - os.system("wget jsonplaceholder.typicode.com/posts/1 -P /tmp") do I need to create deployment package to use bash command in os.system("bash command")?

franklinsijo Over a year ago

@Navjot I have never checked whether wget works in lambda. You can make use of urllib module otherwise. urllib.urlretrieve("https://jsonplaceholder.typicode.com/posts/1", "file.json")

liferacer Over a year ago

Yeah It works with urllib. Thanks for your suggestion. I think I will have to deploy wget and awscli as part of deployment package.

pankajanand18 Over a year ago

@liferacer : how did you add wget to deployment package ?

jamylak Over a year ago

@pankajanand18 He didn't, just used urllib instead

Collectives™ on Stack Overflow

How to execute bash commands in aws lambda

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related