Error connecting AWS ElasticSearch Instance from AWS Lambda function while using Serverless Framework

Question

I am using serverless framework to implement a serverless project. I added some config in my serverless.yml file to create instance for aws elasticsearch service, which is successfully created. Then i created a elasticsearch client in my handler and ping it for testing. So when i ping the elastic search client from serverless offline on my local system, It worked fine and got a 'true' response, but when i deployed the same code on aws lambda than it gets timed out after getting no response till 30 sec.

Have given all policies that can be required but getting no luck.

Serverless.yml:-

ElasticSearchInstance:
  Type: AWS::Elasticsearch::Domain
  Properties:
    EBSOptions:
      EBSEnabled: true
      VolumeType: gp2
      VolumeSize: 10
    ElasticsearchClusterConfig:
      InstanceType: t2.small.elasticsearch
      InstanceCount: 1
      DedicatedMasterEnabled: false
      ZoneAwarenessEnabled: false
    ElasticsearchVersion: 7.1

Handler.js:-

var {Client} = require('elasticsearch');
var client = new Client({
 host: 'Aws elasticsearch endpoint',
 log: 'trace'
});

module.exports.elasticSearchPing = async () => {
  try {
  console.log('Inside elasticSearchPing function!!!!');
  const res = await client.ping({requestTimeout: 900000});
  console.log('Res: ', res);
  return {
  statusCode: 200,
  body: JSON.stringify({ message: 'Connection successful with elasticSearch.' })
 }
} catch (err) {
  console.log('err: ', err);
  return {
    statusCode: err.statusCode || 500,
    headers: { 'Content-Type': 'text/plain' },
    body: 'Error connecting elasticsearch.'
    }
 }
}

Ashish Modi · Accepted Answer · 2020-02-13 17:12:59Z

0

This way there is no way for aws elasticsearch to know who you are. You need to sign your request before sending it to aws elasticsearch. You can use a package called http-aws-es which basicsally reads the aws credentials from your ec2/lambda and signs the request for you. Your code will look like this

const {Client} = require("elasticsearch");
const esConnectionClass = require("http-aws-es");

const client = new Client({
  "host": "Aws elasticsearch endpoint",
  "log": "trace",
  "connectionClass": esConnectionClass
});

module.exports.elasticSearchPing = async () => {
  try {
    console.log("Inside elasticSearchPing function!!!!");
    const res = await client.ping({"requestTimeout": 900000});
    console.log("Res: ", res);
    return {
      "statusCode": 200,
      "body": JSON.stringify({"message": "Connection successful with elasticSearch."})
    };
  } catch (err) {
    console.log("err: ", err);
    return {
      "statusCode": err.statusCode || 500,
      "headers": {"Content-Type": "text/plain"},
      "body": "Error connecting elasticsearch."
    };
  }
};

answered Feb 13, 2020 at 17:12

Ashish Modi

7,7902 gold badges23 silver badges36 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Bilal Khan Over a year ago

I have also tried with the http-aws-es module but it does'nt solves the issue. Actually the problem was with the VPC subnetIds, I am providing the securityGroupIds and subnetIds but when i comment out the subnetIds, it starts working fine, though all my subnetIds are public. Still wonders why this is happenning. Following are the vpc details:- vpc: securityGroupIds: - ${self:custom.secrets.SECURITY_GROUP_ID} # subnetIds: # - ${self:custom.secrets.SUBNET1_ID} # - ${self:custom.secrets.SUBNET2_ID} # - ${self:custom.secrets.SUBNET3_ID}

Ashish Modi Over a year ago

Is ur es in vpc or available outside vpc as well?

Bilal Khan Over a year ago

Using VPC for ES

Collectives™ on Stack Overflow

Error connecting AWS ElasticSearch Instance from AWS Lambda function while using Serverless Framework

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related