AWS Lambda: An error occurred (403) when calling the HeadObject operation: Forbidden

Question

I have a lambda function making a s3 HeadObject call. Even though there has been a custom policy made for this lambda function, I keep getting an 403 error whenever this HeadObject call is made. There is specifically;

An error occurred (403) when calling the HeadObject operation: Forbidden

My policy very clearly allows GetObject calls for the bucket in question. Not sure what the problem is. I have triple check that it all lines up. The line in question is:

    s3 = boto3.client('s3')
    local_file_path = '/tmp/' + key_name.split('/')[-1] + '_REMOTE.json'
    response = s3.head_object(Bucket=environ['OUTPUT_BUCKET'], Key=OUTPUT_FILE_NAME)

Let me know if I can provide more info to help

@b.b3rn4rd the file doesn't exist yet, but does that mean it will return a error? For context I'm implementing this file given by amazon github.com/awslabs/aws-waf-security-automations/blob/master/… — Joseph hooper
– Joseph hooper, Commented Sep 26, 2018 at 0:09

b.b3rn4rd · Accepted Answer · 2018-09-26 00:36:41Z

21

Since you said the file is missing its an expected behaviour if you're missing s3:ListBucket permissions

You need the s3:GetObject permission for this operation. For more information, go to Specifying Permissions in a Policy in the Amazon Simple Storage Service Developer Guide. If the object you request does not exist, the error Amazon S3 returns depends on whether you also have the s3:ListBucket permission.

If you have the s3:ListBucket permission on the bucket, Amazon S3 will return a HTTP status code 404 ("no such key") error.

If you don’t have the s3:ListBucket permission, Amazon S3 will return a HTTP status code 403 ("access denied") error.

Link to the doco

There is also a "different" eventual consistency behaviour for doing HEAD before uploading the object

Amazon S3 Data Consistency Model Amazon S3 provides read-after-write consistency for PUTS of new objects in your S3 bucket in all regions with one caveat. The caveat is that if you make a HEAD or GET request to the key name (to find if the object exists) before creating the object, Amazon S3 provides eventual consistency for read-after-write.

edited Sep 26, 2018 at 0:36

answered Sep 26, 2018 at 0:10

b.b3rn4rd

8,8602 gold badges50 silver badges59 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Joseph hooper Over a year ago

Okay I'll add that permission and give it another try. Thanks!

Joseph hooper Over a year ago

I'm not sure this was the solution. I just got it to run when I created everything in my test environment. For some reason it only works half the time and I'm not understanding why.

Joseph hooper Over a year ago

I'm thinking it might be a regional thing? My test environment succeeded in us-west-1, but I'm having problems in us-east-1. Not sure why that would matter though.

Joseph hooper Over a year ago

I was essentially chasing a red herring on this issue. You were correct in that it was an expected error. I was incorrectly associating it with errors that cloudwatch was reporting to me.

Onevarez · Accepted Answer · 2020-08-26 19:32:17Z

7

I found the solution on this link, so all the credit goes to the author.

Basically you might need to check you have the right permissions ( s3:GetObject and s3:ListBucket at least ) for the operation and the resource expression matches the ARN plus a path e.g this -> "Resource":"arn:aws:s3:::BUCKET_NAME/*" rather than this -> "Resource":"arn:aws:s3:::BUCKET_NAME".

answered Aug 26, 2020 at 19:32

Onevarez

1,15416 silver badges26 bronze badges

Comments

nadavvadan · Accepted Answer · 2021-05-27 11:23:57Z

1

I have just encountered this issue for keys with square brackets (e.g [x][y]abcd.ext).
Upon renaming the keys, everything proceeded to work correctly - 403 error resolved.

That was an incredibly non-intuitive solution.

answered May 27, 2021 at 11:23

nadavvadan

4,2001 gold badge20 silver badges29 bronze badges

Comments

Pharaj Ali · Accepted Answer · 2023-03-01 11:40:40Z

0

s3:ListBucket is a bucket specific permission so iamInlinePolicies should be like this

    Action:
      - s3:ListBucket
      - s3:PutObject
      - s3:GetObject
      - s3:DeleteObject
    Resource: 'arn:aws:s3:::BUCKET_NAME/*'
  - Effect: Allow
    Action:
      - s3:ListBucket
    Resource: 'arn:aws:s3:::BUCKET_NAME'

adding resource like this arn:aws:s3:::BUCKET_NAME/* will provide you to perform operation inside the bucket and for ListBucket, resource are only the bucket arn:aws:s3:::BUCKET_NAME

answered Mar 1, 2023 at 11:40

Pharaj Ali

7516 silver badges6 bronze badges

Collectives™ on Stack Overflow

AWS Lambda: An error occurred (403) when calling the HeadObject operation: Forbidden

4 Answers 4

4 Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

4 Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related