(JavaScript) Synchronizing forEach Loop with callbacks inside

Question

I am doing some computations inside a double forEach Loop something like this:

array.forEach(function(element){
    Object.keys(element).forEach(function(key){

        /* some complex computations with asynchronous callbacks  */        

    });
});

someFunctionHere();

Is there a way for the Loop to finish first before doing the someFunctionHere( ) function? or any way that the program will know if the Loop is finish before continuing to someFunctionHere( ) ...

I may be missing some forums but the ones I found did not helped me of what I want to achieve and by the way I am doing this in NodeJS , also I am asking if there are existing libraries that can make this happen.

I forgot to Add this up or should this be an another question?

Is there a way to do the iteration synchronously that it will only proceed to the next iteration once the current iteration is done? (Sorry for this)

Thanks for any help...

Yes, it's basically 100% guaranteed that "someFunctionHere" will be called before the asynchronous operations are finished. — Pointy
– Pointy, Commented Mar 1, 2013 at 16:15
Regarding your edit: Golo's answer shows how you can use async.js to do what you want. (I believe that Clumpy.js can also be used for this.) My answer shows how you can "roll your own" solution if, for some reason, you cannot (or simply do not want to) use third-party libraries. — Ted Hopp
– Ted Hopp, Commented Mar 1, 2013 at 21:18
I have done the above problem using async.forEach() but it messed up my database since iterations are not called one iteration at a time(or if the iterations are done synchronously I might have a logical error in the code), so Golo's answer might do the same thing as I have done in asyn.forEach(), I mean from the term 'async' do iterations are not done synchronously? — megamoth
– megamoth, Commented Mar 2, 2013 at 0:24
Or it could be done one at a time but does not wait for the current iteration to finish before proceeding to the next iteration, am I right? — megamoth
– megamoth, Commented Mar 2, 2013 at 0:36

Golo Roden · Accepted Answer · 2013-03-02 13:15:37Z

46

Take a look at async.js, and especially its control flow statements, such as each whilst and until.

Using async.js you can get what you want to have.

In your actual situation what you want is the each function (which has formerly been known as forEach), respectively the eachSeries function which does not run the single iterations in parallel, but in serial (see the documentation for eachSeries for details).

To provide an example:

async.eachSeries([ 2, 3, 5, 7, 11 ], function (prime, callback) {
  console.log(prime);
  callback(); // Alternatively: callback(new Error());
}, function (err) {
  if (err) { throw err; }
  console.log('Well done :-)!');
});

This will iterate over the array of prime numbers and print them in the correct order, one after each other, and finally print out Well done :-)!.

edited Mar 2, 2013 at 13:15

answered Mar 1, 2013 at 16:17

Golo Roden

152k102 gold badges318 silver badges445 bronze badges

Sign up to request clarification or add additional context in comments.

11 Comments

megamoth Over a year ago

Thanks for this one , but using the async library is not an option , I mean each iteration in the loop should really be synchronous since it is doing intensive db operations that it should only be one iteration at a time and I forgot to put this in the question also sorry for that

Golo Roden Over a year ago

The sample exactly shows this behavior: Each iteration within the loop runs synchronous. It's perfectly serialized.

megamoth Over a year ago

I have used async.forEach() in my previous solution before asking the question , so the iterations in async are done one at a time? because if so , I might have some logical errors in my code that I did not saw or it could be done one at a time but does not wait on the current iteration to finish before proceeding to the next iteration

Golo Roden Over a year ago

I guess you mean async.each(). This one runs all the tasks in parallel, but there is also async.eachSeries(), which should be perfectly fine for what you want to do. Check out github.com/caolan/async#eachseriesarr-iterator-callback for details.

megamoth Over a year ago

stackoverflow.com/questions/10390041/… , this is the async.forEach() that I am talking about , maybe this is not really the solution but the description of the link that you gave me is what I want.

|

Ted Hopp · Accepted Answer · 2013-03-01 20:52:33Z

7

You can wrap your callbacks in a count-down closure:

var len = array.length;

function countdownWrapper(callback, callbackArgs) {
    callback(callbackArgs);
    if (--len == 0) {
        someFunctionHere();
    }
}

array.forEach(function(element){
    Object.keys(element).forEach(function(key){

        var wrappedCallback = countdownWrapper.bind(callback);
        /* some complex computations with asynchronous WRAPPED callbacks  */

    });
});

If the call-backs have different number of arguments, you can do a little surgery on arguments instead of using an explicit callbackArgs parameter.

EDIT Your edit clarifies that you want to start each complex computation after the previous calculation completes it's callback. This can also be easily arranged through closures:

function complexOp(key, callback) { /* . . . */ }

function originalCallback(...) { /* . . . */ }

function doSomethingElse() { /* . . . */ }

var iteratorCallback = (function (body, callback, completion) {
    var i = 0, len = array.length;
    return function iterator() {
        callback.apply(null, arguments);
        if (++i < len) {
            body(array[i], iterator);
        } else {
            completion();
        }
    };
}(complexOp, originalCallback, doSomethingElse)); 

// kick everything off:
complexOp(array[0], iteratorCallback);

edited Mar 1, 2013 at 20:52

answered Mar 1, 2013 at 16:22

Ted Hopp

235k48 gold badges412 silver badges533 bronze badges

9 Comments

megamoth Over a year ago

This is really of great help it might solve one of my problems and I still have to check this out , thanks!

Golo Roden Over a year ago

This basically is nothing else but what async.js gives you out of the box. It's just not "standardized", but "proprietary".

Ted Hopp Over a year ago

@GoloRoden - Proprietary? That's preposterous. There are plenty of examples of this trick on the web, and it's easy enough to come up with it on the fly. (For the record, I did not copy the code in my post from anywhere.) The point of the post is that you don't need an entire package like async.js or Clumpy.js; it's just a few lines of code.

Golo Roden Over a year ago

That's why I put it in quotes. Of course it's not proprietary in its original meaning, but why re-invent the wheel for the 1000th time? And of course yo can do it on your own instead of using a library, but you can also do everything still in assembler. There's simply no meaningful reason why you would want to do it on your own.

Ted Hopp Over a year ago

@GoloRoden - Okay, thanks for the clarification. Your first comment sounded like an accusation of some sort; perhaps I was being overly sensitive. Of course using a package like async.js or Clumpy.js is the preferred approach. However, there's a learning curve involved in using any third-party library. Also, sometimes including third-party libraries in your code can be problematic for non-technical reasons. It's always good to know how to do these things yourself.

|

Mohit Chawla · Accepted Answer · 2016-03-11 10:02:52Z

HIGH PERFORMANCE SOLUTION:

There might be a case when you might want/allow to process the array asynchronously/parallely but want a function to be called after all members of forEach have been processed. Example:

var async = require('async');

async.forEach(array,function(elementOfArray, callback){
    //process each element of array in parallel 
    // ..
    // ..
    // .. 
    callback(); //notify that this iteration is complete
}, function(err){
 if(err){throw err;}
 console.log("processing all elements completed");
});

Hence, this way you perform non-blocking CPU intensive operations.

NOTE: When you use eachSeries for huge arrays, so many iterative callbacks might overflow the stack.

Read here: https://github.com/caolan/async#control-flow

jony89 · Accepted Answer · 2017-06-19 10:05:39Z

2

For browsers which support Promise (or using polyfill) / nodejs I've implemented my self a sync version of forEach with callbacks, so I'll just share that here..

The callback must return a promise..

function forEachSync(array, callback) {
    let lastIndex = array.length - 1;
    let startIndex = 0;

    return new Promise((resolve, reject) => { // Finish all
        let functionToIterateWith = (currIndex) => {
            if (currIndex > lastIndex) {
                return resolve();
            } else {
                callback(array[currIndex]).then(() => {
                    functionToIterateWith(currIndex + 1);
                }).catch(err => reject(err));
            }
        }

        functionToIterateWith(startIndex);
    });
}

edited Jun 19, 2017 at 10:05

answered Jun 19, 2017 at 8:52

jony89

5,3863 gold badges33 silver badges42 bronze badges

Comments

Erkam KUCET · Accepted Answer · 2018-09-05 20:55:38Z

2

let dataObject = {};
Promise.all(objectArray.map(object => {
        return new Promise(resolve => {
            yourFunction(object, anotherParameter).then(returnValue => {
                dataObject[object] = returnValue;
                resolve();
            });
        });
    })).then(() => {
        return dataObject;
    });

answered Sep 5, 2018 at 20:55

Erkam KUCET

5158 silver badges25 bronze badges

Collectives™ on Stack Overflow

(JavaScript) Synchronizing forEach Loop with callbacks inside

5 Answers 5

11 Comments

9 Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

11 Comments

9 Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related