Delete all the objects in the AWS-S3 bucket that have key with specific pattern?

Question

I want to delete all the objects from an s3 bucket that has the key with the certain pattern. My questions??

Can I simply pass regular expression on the key value as illustrated in the code below?
One option would be to list all the objects and filter the keys that matches the regular expression and add them all to the deleteObjects method. I find this inefficient. So, any other way around? Would be a great help? Thanks in advance...

var params = {
  Bucket: 'sample_bucket', 
  Delete: { 
    Objects: [ 
      {
        Key: '/video-044567/g' // here i want something like that
      },
      
    ],
  },
};

s3.deleteObjects(params, function(err, data) {
  if (err) console.log(err, err.stack); 
  else     console.log(data);           
});

Please accept my correct answer, even if you don't like that the answer is "no you can't" — Megan D
– Megan D, Commented Jul 9, 2018 at 15:57

ggcarmi · Accepted Answer · 2019-03-17 09:42:08Z

3

you can do it using aws cli : https://aws.amazon.com/cli/ and some unix command.

this aws cli commands should work:

aws s3 rm selina-data-lake-test --exclude "*" --include "<your_regex>"

if you want to include sub-folders you should add the flag --recursive

or with unix commands:

aws s3 ls s3://<your_bucket_name>/ | awk '{print $4}' | xargs -I%  <your_os_shell>   -c 'aws s3 rm s3:// <your_bucket_name>  /% $1'

explanation: list all files on the bucket --pipe--> get the 4th parameter(its the file name) --pipe--> run delete script with aws cli

answered Mar 17, 2019 at 9:42

ggcarmi

4884 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Jan · Accepted Answer · 2022-02-03 21:32:47Z

0

Assuming you are using the aws-sdk NPM module, AWS.S3 class does not support recursive delete of objects.

If you really don't want to perform multiple operations to delete each object, consider using AWS CLI which supports --recursive delete based on folder prefix. Theoretically you could execute aws cli within Node JS depending on how your system credentials are configured.

https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3/rm.html

edited Feb 3, 2022 at 21:32

Jan

5831 gold badge5 silver badges17 bronze badges

answered May 4, 2018 at 18:21

Megan D

3853 silver badges13 bronze badges

2 Comments

Sandesh Poudel Over a year ago

Well, thanks but i have more than 20 files for a specific video and i want to delete those files when user request to delete the video. So, can’t use CLI.

John Rotenstein Over a year ago

@SandeshPoudel In that case, you will have to list the files and then request the specific files to be deleted. You might consider it inefficient, but there is no wildcard delete capability in Amazon S3.

Collectives™ on Stack Overflow

Delete all the objects in the AWS-S3 bucket that have key with specific pattern?

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related