Access S3 from Lambda within VPC using Terraform

Question

I have a Lambda

resource "aws_lambda_function" "api" {
  function_name = "ApiController"

  timeout = 10

  s3_bucket = "mn-lambda"
  s3_key = "mn/v1.0.0/sketch-avatar-api-1.0.0-all.jar"

  handler = "io.micronaut.function.aws.proxy.MicronautLambdaHandler"
  runtime = "java11"

  memory_size = 1024

  role = aws_iam_role.api_lambda.arn

  vpc_config {
    security_group_ids = [aws_security_group.lambda.id]
    subnet_ids = [for subnet in aws_subnet.private: subnet.id]
  }
}

Within a VPC

resource "aws_vpc" "vpc" {
  cidr_block = var.vpc_cidr_block
  enable_dns_support = true
  enable_dns_hostnames = true
}

I created a aws_vpc_endpoint because I read that's what's need for my VPC to access S3

resource "aws_vpc_endpoint" "s3" {
  vpc_id = aws_vpc.vpc.id
  service_name = "com.amazonaws.${var.region}.s3"
}

I created and attached a policy allowing access to S3

resource "aws_iam_role_policy_attachment" "s3" {
  role = aws_iam_role.api_lambda.name
  policy_arn = aws_iam_policy.s3.arn
}

resource "aws_iam_policy" "s3" {
  policy = data.aws_iam_policy_document.s3.json
}

data "aws_iam_policy_document" "s3" {
  statement {
    effect = "Allow"
    resources = ["*"]

    actions = [
      "s3:*",
    ]
  }
}

It might be worth noting that the buckets I'm trying to access is created using the aws cli but in the same region. So not with terraform.

The problem is that my Lambda is timing out when I try to read files from S3.

The full project can be found here should anyone want to take a peek.

I guess it's more of a connectivity issue rather than a code issue. Can you read S3 buckets created via the CLI? — harshavmb
– harshavmb, Commented Nov 26, 2020 at 21:16
Is the aws_vpc_endpoint attached to the VPC's route table? — Montgomery Watts
– Montgomery Watts, Commented Nov 27, 2020 at 0:03
You forgot to specify route_table_ids. S3 is VPC gateway and it must be associated with appropriate route tables. — Marcin
– Marcin, Commented Nov 27, 2020 at 0:05

Marcin · Accepted Answer · 2020-11-27 00:26:16Z

3

You are creating com.amazonaws.${var.region}.s3 which is gateway VPC endpoint , which shouldn't be confused with interface VPC endpoints.

One of the key differences between the two is that the gateway type requires association with route tables. Thus you should use route_table_ids to associate your S3 gateway with route tables of your subnets.

For example, to use default main VPC route table:

resource "aws_vpc_endpoint" "s3" {
  vpc_id = aws_vpc.vpc.id
  service_name = "com.amazonaws.${var.region}.s3"

  route_table_ids = [aws_vpc.vpc.main_route_table_id]
}

Alternatively, you can use aws_vpc_endpoint_route_table_association to do it as well.

answered Nov 27, 2020 at 0:26

Marcin

241k16 gold badges315 silver badges368 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Access S3 from Lambda within VPC using Terraform

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related