57

I am following these instructions to create a basic web scraper that executes in Lambda. I have experience writing selenium code, but not with Node JS. I got the project running in Lambda, but when I tried editing the project locally in order to execute the selenium code I want, It doesn't work. Anything in the exports.handler doesn't get executed when I run node index.js. How would I execute this project locally? Thanks!

1
  • You need to call your function, not just export the declaration. Import your exported handler.js assigned to a variable in index.js and call your function for example. (Assuming exports.handler is a function.) Commented Aug 25, 2018 at 16:13

8 Answers 8

84

This is what I did:

index.js

exports.handler = async (event) => {
    console.log('hello world');

    const response = {
        statusCode: 200,
        body: JSON.stringify('Hello from Lambda!')
    };

    return response;
};

package.json

// using require
"scripts": {
  "locally": "node -e \"console.log(require('./index').handler(require('./event.json')));\""
}

// or the following for ESM using import
"scripts": {
  "locally": "node --input-type=module -e \"import {handler} from './index.mjs'; console.log(await handler(JSON.parse(fs.readFileSync('./event.json'))));\""
}

event.json

{
  "Records": [
    {
      "eventVersion": "2.0",
      "eventSource": "aws:s3",
      "awsRegion": "eu-central-1",
      "eventTime": "1970-01-01T00:00:00.000Z",
      "eventName": "ObjectCreated:Put",
      "userIdentity": {
        "principalId": "AIDAJDPLRKLG7UEXAMPLE"
      },
      "requestParameters": {
        "sourceIPAddress": "127.0.0.1"
      },
      "responseElements": {
        "x-amz-request-id": "C3D13FE58DE4C810",
        "x-amz-id-2": "FMyUVURIY8/IgAtTv8xRjskZQpcIZ9KG4V5Wp6S7S/JRWeUWerMUE5JgHvANOjpD"
      },
      "s3": {
        "s3SchemaVersion": "1.0",
        "configurationId": "testConfigRule",
        "bucket": {
          "name": "my-bucket",
          "ownerIdentity": {
            "principalId": "A3NL1KOZZKExample"
          },
          "arn": "arn:aws:s3:::my-bucket"
        },
        "object": {
          "key": "HelloWorld.jpg",
          "size": 1024,
          "eTag": "d41d8cd98f00b204e9800998ecf8427e",
          "versionId": "096fKKXTRTtl3on89fVO.nfljtsv6qko"
        }
      }
    }
  ]
}

Shell

npm run locally

Output

> node -e "console.log(require('./index').handler({}));"

hello world
Promise { { statusCode: 200, body: '"Hello from Lambda!"' } }
Sign up to request clarification or add additional context in comments.

6 Comments

Why the extra npm step? It's more straightforward to just run the node command, right?
The extra npm step is not necessary. It is just a convenience for me, because I could add arguments to the call that are specific to that code base. And quite frankly, it is easier for me to remember :)
How can we pass events in this case?
You can create a new json file called sad-event.json and require that file rather than "./event.json". You could organize them in a folder called test-events, so your path could be "test-events/happy.json"
for the new ESM hotness: node --env-file=.env --input-type=module -e "import {handler} from './index.mjs'; console.log(await handler(JSON.parse(fs.readFileSync('./event.json'))));"
|
52

You need to call your handler function from another file lets say testHandler.js in order to run via NodeJs.

This will be done like this

//import your handler file or main file of Lambda
let handler = require('./handler');

//Call your exports function with required params
//In AWS lambda these are event, content, and callback
//event and content are JSON object and callback is a function
//In my example i'm using empty JSON
handler.handlerEvent( {}, //event
    {}, //content
    function(data,ss) {  //callback function with two arguments 
        console.log(data);
    });

Now you can use node testHandler.js to test your handler function.

EDIT: Sample Event and content data as requested

Event:

{
    "resource": "/API/PATH",
    "path": "/API/PATH",
    "httpMethod": "POST",
    "headers": {
        "Accept": "*/*",
        "Accept-Encoding": "gzip, deflate, br",
        "Accept-Language": "en-GB,en-US;q=0.9,en;q=0.8",
        "cache-control": "no-cache",
        "CloudFront-Forwarded-Proto": "https",
        "CloudFront-Is-Desktop-Viewer": "true",
        "CloudFront-Is-Mobile-Viewer": "false",
        "CloudFront-Is-SmartTV-Viewer": "false",
        "CloudFront-Is-Tablet-Viewer": "false",
        "CloudFront-Viewer-Country": "IN",
        "content-type": "application/json",
        "Host": "url.us-east-1.amazonaws.com",
        "origin": "chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop",
        "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36",
        "Via": "2.0 XXXXXXXXXXXXXX.cloudfront.net (CloudFront)",
        "X-Amz-Cf-Id": "XXXXXXXXXX51YYoOl75RKjAWEhCyna-fuQqEBjSL96TMkFX4H0xaZQ==",
        "X-Amzn-Trace-Id": "Root=1-XXX03c23-25XXXXXX948c8fba065caab5",
        "x-api-key": "SECUREKEY",
        "X-Forwarded-For": "XX.XX.XXX.XXX, XX.XXX.XX.XXX",
        "X-Forwarded-Port": "443",
        "X-Forwarded-Proto": "https"
    },
    "multiValueHeaders": {
        "Accept": [ "*/*" ],
        "Accept-Encoding": [ "gzip, deflate, br" ],
        "Accept-Language": [ "en-GB,en-US;q=0.9,en;q=0.8" ],
        "cache-control": [ "no-cache" ],
        "CloudFront-Forwarded-Proto": [ "https" ],
        "CloudFront-Is-Desktop-Viewer": [ "true" ],
        "CloudFront-Is-Mobile-Viewer": [ "false" ],
        "CloudFront-Is-SmartTV-Viewer": [ "false" ],
        "CloudFront-Is-Tablet-Viewer": [ "false" ],
        "CloudFront-Viewer-Country": [ "IN" ],
        "content-type": [ "application/json" ],
        "Host": [ "apiurl.us-east-1.amazonaws.com" ],
        "origin": [ "chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop" ],
        "User-Agent": [ "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" ],
        "Via": [ "2.0 XXXXXXXXXXXXXX.cloudfront.net (CloudFront)" ],
        "X-Amz-Cf-Id": [ "XXXXXXXXXhCyna-fuQqEBjSL96TMkFX4H0xaZQ==" ],
        "X-Amzn-Trace-Id": [ "Root=1-XXXXXXX67339948c8fba065caab5" ],
        "x-api-key": [ "SECUREAPIKEYPROVIDEDBYAWS" ],
        "X-Forwarded-For": [ "xx.xx.xx.xxx, xx.xxx.xx.xxx" ],
        "X-Forwarded-Port": [ "443" ],
        "X-Forwarded-Proto": [ "https" ]
    },
    "queryStringParameters": null,
    "multiValueQueryStringParameters": null,
    "pathParameters": null,
    "stageVariables": null,
    "requestContext": {
        "resourceId": "xxxxx",
        "resourcePath": "/api/endpoint",
        "httpMethod": "POST",
        "extendedRequestId": "xxXXxxXXw=",
        "requestTime": "29/Nov/2018:19:21:07 +0000",
        "path": "/env/api/endpoint",
        "accountId": "XXXXXX",
        "protocol": "HTTP/1.1",
        "stage": "env",
        "domainPrefix": "xxxxx",
        "requestTimeEpoch": 1543519267874,
        "requestId": "xxxxxxx-XXXX-xxxx-86a8-xxxxxa",
        "identity": {
            "cognitoIdentityPoolId": null,
            "cognitoIdentityId": null,
            "apiKey": "SECUREAPIKEYPROVIDEDBYAWS",
            "cognitoAuthenticationType": null,
            "userArn": null,
            "apiKeyId": "xxXXXXxxxxxx",
            "userAgent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36",
            "accountId": null,
            "caller": null,
            "sourceIp": "xx.xxx.xxx.xxx",
            "accessKey": null,
            "cognitoAuthenticationProvider": null,
            "user": null
        },
        "domainName": "url.us-east-1.amazonaws.com",
        "apiId": "xxxxx"
    },
    "body": "{\n    \"city\": \"Test 1 City\",\n    \"state\": \"NY\",\n    \"zipCode\": \"11549\"\n}",
    "isBase64Encoded": false
}

Content:

{
    "callbackWaitsForEmptyEventLoop": true,
    "logGroupName": "/aws/lambda/lambda-name",
    "logStreamName": "2018/11/29/[$LATEST]xxxxxxxxxxxb",
    "functionName": "lambda-name",
    "memoryLimitInMB": "1024",
    "functionVersion": "$LATEST",
    "invokeid": "xxxxx-xxx-11e8-xxx-xxxxxxxf9",
    "awsRequestId": "xxxxxx-xxxxx-11e8-xxxx-xxxxxxxxx",
    "invokedFunctionArn": "arn:aws:lambda:us-east-1:xxxxxxxx:function:lambda-name"
}

3 Comments

Can you show an example of what event usually looks like when called on AWS Lambda?
@PetrusTheron added sample event and content.
The events look different depending on the trigger for the event. In the web UI to edit a lambda, the test event selection dropdown has "Configure Test Event" as one of the choices. In there, they offer prototype events for all the types of triggers.
3

Perhaps the simplest way to get started, after some testing (with Node 14.17.3)

let handler = require('./index.js');

handler.handler ( 
  {}, // event
  {}, // content 
  (error, result) => { 
     if (error) console.error(JSON.stringify(error, null, 2));
     else console.log(JSON.stringify(result, null, 2));
  }
);

Comments

2

In your index.js, just defined and exported a handler function, but no one calls it. In the Lambda environment, some AWS code will call this handler with message. In your local environment, you have to call your handler by yourself.

You could also have a look of this doc, it is a way to "simulate" Lambda in local environment.

Comments

2

You can check out lambda-local. It's a little fancier than the accepted answer above. For example, it supports passing environment variables and using JSON files for your payloads.

Comments

2

If you want just to execute it locally you can use the official sam cli tool https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-local-invoke.html

If you use VSCode you can also check out this extension:

https://marketplace.visualstudio.com/items?itemName=bogdan-onu.invoke

Comments

2

Here I am giving a solution for the general case of an synchronous call.

Function signature in your entrypoint index.js:
exports.handler = function(event, context, callback) {

  1. Create a caller file, e.g. testIndex.js
  2. Give it the following contents:
    /**
    This is a caller for ./index.js
    */
    const thatIndex = require('./index');

    const EVENT = {
      "somekey": {
        "somesubkey": "somevalue"
      }
    }

    thatIndex.handler (
        // event
        EVENT,
        // context
        {},
        // callback function with two arguments
        function(err, payload) {
            console.log(err);
            console.log(payload);
        }
    );
  1. Go back to your node console (Docker container suggested)
  2. You may use environment variables, if your index.js needs them: define each with export MYVAR1=myvalue1
  3. Run your caller: node testIndex.js
  4. The console should output the payload, provided your main file (entrypoint) promise call is chained-up with: (example)
    .then(() => callback(null, {
       statusCode: 301,
       headers: {'location': process.env.URL + '/' + somekey },
       body: '',
       })
    )
    .catch(err => callback(err))

Tip: you may need to clean up the cache with node clean cache if you notice that it's the previous code that's called.

Other Tip: in the AWS environment, you don't need to zip up the node_modules/aws-sdk folder, as it will be part of the standard execution context. But on your test machine you will need to. So, in development mode, simple run npm install, not npm install --omit=dev

So, before zipping the folder that contains your function, just remove node_modules folder recursively, and rebuild the dependencies with npm install --ommit=dev

Example of zipping the folder: zip -r ..\resizeImage.zip .

This is what your package.json file could contain:

  "dependencies": {
    "sharp": "^0.27.2"
  },
  "devDependencies": {
    "aws-sdk": "^2.1195.0"
  }

Comments

-2

IMO. The best way to test a lambda is to actually just test it! What does that mean? Will simply use some testing library (like jest for example) and simply create a test on your handler function. Mock anything you need and provide some data you expect that will come into the lambda (for the event and context if needed). And that's it. You have some tests written and a quick way to test your lambda at the same time.

1 Comment

And if you're deploying your lambda using terraform for example I could recommend a packager that supports TypeScript for lambdas that I built. npmjs.com/package/funpack

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.