How can I upload files to the replicate API with Node JS

Question

I'm trying to use the sadTalker AI model through replicate.

The point of sadtalker is that you upload an image of a person and an audio file of someone talking, and it gives you back a deepfake video of someone talking.

I've setup a basic environment to do this. All I have is a .env file storing my API key, two files called deepfakeAudio.mp3 and deepfakeImage.jpg.

My code is as follows:

import Replicate from "replicate";

import dotenv from "dotenv";
import fs from "fs";

dotenv.config();

fs.readFile("./deepfakeImage.jpg", (error, inputImage) => {
  fs.readFile("./deepfakeAudio.mp3", (error, inputAudio) => {
    getVideo();
  });
});

async function getVideo(inputImage, inputAudio) {
  const replicate = new Replicate({
    auth: process.env.REPLICATE_API_TOKEN,
  });
  const output = await replicate.run(
    "cjwbw/sadtalker:3aa3dac9353cc4d6bd62a8f95957bd844003b401ca4e4a9b33baa574c549d376",
    {
      input: {
        source_image: inputImage,
        driven_audio: inputAudio
      }
    }
  );
  console.log(output);
}

When I try to run this, I get the error:

throw new Error(`API request failed: ${response.statusText}`);
            ^

Error: API request failed: Unprocessable Entity

The problem is that I don't know how I'm supposed to load the image and audio into the API.

To load the image and audio, I've tried 2 things:

The first time, I tried to just input the path of the audio and image. This gave me an error.

The second time, as shown in the code, I tried to load the audio and image using fs, I still get an error.

Has anyone experienced this issue before?

This is very confusing because I have no idea of what I'm actually supposed to do.

Abhishek Kaushik · Accepted Answer · 2023-06-18 16:17:48Z

5

The replicate docs mention that replicate expects file input as base64 encoded data URI.

So your modified code would look something like this :

import Replicate from "replicate";

import dotenv from "dotenv";
import { promises as fs } from "fs";

dotenv.config();

const data = await fs.readFile("./deepfakeImage.jpg", "utf-8");
const base64 = data.toString("base64");
const mimeType = "image/jpg";
const dataURI = `data:${mimeType};base64,${base64}`;


async function getVideo(inputImage, inputAudio) {
  const replicate = new Replicate({
    auth: process.env.REPLICATE_API_TOKEN,
  });
  const output = await replicate.run(
    "cjwbw/sadtalker:3aa3dac9353cc4d6bd62a8f95957bd844003b401ca4e4a9b33baa574c549d376",
    {
      input: {
        source_image: dataURI,
        driven_audio: inputAudio
      }
    }
  );
  console.log(output);
}

answered Jun 18, 2023 at 16:17

Abhishek Kaushik

1172 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

patrick Over a year ago

The docs don't specify 'utf-8' -- I needed to remove that for it to run in the readFile call

tobek Mar 12 at 2:50

Note that the docs (replicate.com/docs/topics/predictions/input-files) only recommend this method for files under 1MB, though it works for me for larger files. Supposed to support sending a Blob, File, or Buffer in, for files up to 100MB, though I couldn't get it to work though, just get a 401 Client Error: Unauthorized for url error for the resulting uploaded file. I manually uploaded the file to the web myself and sent in a URL string and it worked.

Collectives™ on Stack Overflow

How can I upload files to the replicate API with Node JS

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related