How to upload file from URL to S3 bucket (aws-sdk-js v3)?

Question

I've tried the below code with no luck, which showing this error message:

(node:88634) UnhandledPromiseRejectionWarning: NotImplemented: A header you provided implies functionality that is not implemented

import { fromIni } from '@aws-sdk/credential-provider-ini'
import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3'
import https from 'https'
import { Readable } from 'stream'
import awsConfig from './aws-exports'

const s3Client = new S3Client({
  credentials: fromIni({ profile: 'default' }),
  region: 'ap-northeast-1',
})

async function getFileFromUrl(url: string): Promise<Readable> {
  return new Promise((resolve) => {
    https.get(url, (response) => {
      resolve(response)
    })
  })
}

async function upload(file: Readable) {
  const uploadCommand = new PutObjectCommand({
    Bucket: awsConfig.aws_user_files_s3_bucket,
    Key: 'test.jpg',
    Body: file,
    ACL: 'public-read',
  })

  await s3Client.send(uploadCommand)
}

async function migrate() {
  const file = await getFileFromUrl(
    'https://example.com/logo.png'
  )
  await upload(file)
  console.log('done')
}

migrate()

I could confirm that it's ok if I change the Body to a string... Does anyone know how to do this correctly? Thanks!

David Mold · Accepted Answer · 2021-07-25 14:13:31Z

1

The problem here is that your getFileFromUrl function does not work, and AWS doesn't know how to handle the object you are handing it. You need to wait for the data event from https, like this:

async function getFileFromUrl (url) {
  return new Promise((resolve) => {
    https.get(url, (response) => {
      response.on('data', (d) => {
        resolve(d)
      })
    })
  })
}

answered Jul 25, 2021 at 14:13

David Mold

3174 silver badges6 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to upload file from URL to S3 bucket (aws-sdk-js v3)?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related