Handling expensive task(nested for loop) on Node.js

Question

First I write summarized pseudo code.

const ExcelSheet = new ExcelSheet() // excel.js library

const usersWithPlayedGames  = await findAllUsers({ include: GameTable });

for (let i = 0; i < usersWithPlayedGames.length; i++) {
    // Write some user data on Excel.
    ...
    ...
    for (let j = 0; j < usersWithPlayedGames[i].length; j++) {
        // Write some user's game on Excel
        ...
        ...
        for (let k = 0; k < usersWithPlayedGames[i][j].length; k++) {
            // Write some users's game's company data on Excel
            ...
            ...
        }
    }
}

res.send(ExcelSheet.toFile());

The actual code is pretty long.

And it take client req => res time almost 15sec.

I know my problem solution is not good.

I can do refactoring with this code.

But the real problem is it blocks another client request.

I search on google and find several solutions.

Node.js child process spawn.
Make with a callback function(I don't know exactly how to do).
Write more than a good algorithm.

What's the concept I'm missing from Node.js?

Please help me with a small favor.

Thank you.

Write some user data on Excel., Write some user's game on Excel and Write some users's game's company data on Excel .... is there an asynchronous way of doing these? Since you haven't shown this code its hard to guess ... but that would solve the blocks another client request issue — Bravo
– Bravo, Commented Oct 4, 2019 at 1:36
@Bravo Thank you but I can't find with excel.js library async method. — yusung lee
– yusung lee, Commented Oct 4, 2019 at 1:47
There are some, but perhaps the refactoring from your current code is going to be too difficult. — Bravo
– Bravo, Commented Oct 4, 2019 at 1:51

James Hay · Accepted Answer · 2019-10-04 01:42:46Z

1

You're porbably better off running this in a child process or somewhere else outside your web request thread, but if that's not an option, you could break the task up using something like setImmediate

const ExcelSheet = new ExcelSheet() // excel.js library

const usersWithPlayedGames  = await findAllUsers({ include: GameTable });

const excelLoop = (index) => {

  for (let j = 0; j < usersWithPlayedGames[index].length; j++) {
      // Write some user's game on Excel
      ...
      ...
      for (let k = 0; k < usersWithPlayedGames[index][j].length; k++) {
          // Write some users's game's company data on Excel
          ...
          ...
      }
  }

  if (index < usersWithPlayedGames.length) {
    setImmediate(() => excelLoop(index + 1))
  }
  else {
    res.send(ExcelSheet.toFile());
  }
};

excelLoop(0);

Documentation for setImmediate

answered Oct 4, 2019 at 1:42

James Hay

7,3656 gold badges35 silver badges59 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

yusung lee Over a year ago

Thank you very much. I will follow your link

Collectives™ on Stack Overflow

Handling expensive task(nested for loop) on Node.js

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related