How to handle multiple requests for CPU intensive tasks in NodeJS?

Question

In my NodeJS, there is a huge processing task that comprises downloading the audio files from IBM cloud that are around 600MB(8-10 audios each of around 70MB) and then processing those audio files using FFMPEG https://www.npmjs.com/package/ffmpeg. Audio processing takes very huge time for around 5 minutes. This process is asynchronous so my Front-end will be responsive.

How can I handle multiple requests for the same processing of audios but the problem is multiple requests will have different audios to get downloaded and processed, so the race condition might occur while handling multiple requests. By reading other stack overflow answers, I get to know there are approaches like creating child process or worker threads. What will be the appropriate approach to my use-case and how can I make sure race-condition is not met?

That ffmpeg module already spawns a separate process for each function call. That's why it's asynchronous. It's possible that you don't need to do anything at all because each function call is already its own child process. So possibly the best way for you to make your server faster/more responsive is to simply use a CPU with more cores — slebetman
– slebetman, Commented May 19, 2020 at 6:26

slebetman · Accepted Answer · 2020-05-19 06:43:51Z

1

From the documentation of the ffmpeg module we see this:

This library provides a set of functions and utilities to abstract commands-line usage of ffmpeg.

I had a look at the source code of the ffmpeg module and found the core of the implementation is this:

utils.exec([ffmpeg.bin,'-formats','2>&1'], settings, ...

Where utils.exec is basically this:

child_process.exec(finalCommand, settings, ...

So basically the ffmpeg module is already using a child process to perform CPU intensive tasks. Meaning that node.js is not doing any CPU intensive tasks.

Which means you don't need to do anything extra to handle multiple requests. Any additional thing you do will simply use more RAM without any performance benefits.

How to scale:

Scaling this is the same in any language because your server is essentially already multi-threaded/multi-processing. Just use a CPU with more cores.

Scaling beyond CPU:

Big services such as Netflix and YouTube can process tens of thousands of media in parallel. Obviously you cannot buy a CPU with tens of thousands of cores. How you scale is to load-balance your service across multiple CPUs/servers instead of using ffmpeg directly:

 ┌─────────────────┐           ┌─────────────────────┐
 │ node web server │ --------> │ node ffmpeg servers │┐
 └─────────────────┘           └┬────────────────────┘│┐
                                └┬────────────────────┘│
                                 └─────────────────────┘

answered May 19, 2020 at 6:43

slebetman

115k19 gold badges147 silver badges180 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Vijay Singh Kholiya Over a year ago

So Node JS will take care of the simultaneous multiple requests for this service and depending upon the availability of the CPU resources it will assign the task accordingly and will eliminate the race condition, right?

slebetman Over a year ago

Yes. Basically it runs one ffmpeg process for each request. This is similar to opening a command line terminal and typing the ffmpeg command for each request. Basically it's not node.js who is handling it but your OS and it is not node.js who is taking care of it but the programmer who wrote the ffmpeg module. All node.js does is wait for all of them in parallel - this is exactly the same as how node.js can handle multiple parallel network requests.

Vijay Singh Kholiya Over a year ago

I got it instead of creating worker threads for this I should focus on load-balancing to distribute the load. What do you suggest for the load balancing implementation? FYI, my front-end is connected to Node JS using web sockets and for this feature of audio processing I have setup express app.

Rishabh Chakrabarti · Accepted Answer · 2020-05-19 06:14:05Z

1

Background

I had a similar experience, but it was with video transformation and processing. I had to use ffmpeg to transform videos into different container and quality formats.

Solution

The best way to go about this, for my scenario was to create a job queue. I used celery and it was able to handle it very well.

The equivalent for your use case would be - https://github.com/mher/node-celery

You would have to spawn up some workers that would get assigned a task from a message queue database like RabbitMQ. After the task is done, they asynchronously send back a response. Hopefully this works for your scenario

answered May 19, 2020 at 6:14

Rishabh Chakrabarti

113 bronze badges

1 Comment

Vijay Singh Kholiya Over a year ago

Can you share your source code how you implemented it in your application, would be great for me!

Collectives™ on Stack Overflow

How to handle multiple requests for CPU intensive tasks in NodeJS?

2 Answers 2

How to scale:

Scaling beyond CPU:

3 Comments

Background

Solution

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

How to scale:

Scaling beyond CPU:

3 Comments

Background

Solution

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related