Make callback wait till function returns result in node js

Question

There is an array of files that will have to be processed. Files may be of different types, so there are different functions doing the processing. I expect to go through all files, collect logs while at it, and in the end return a variable where all the logs are stored.

The code:

var file_list = req.body.filelist.split(','); //array of filenames
var logs = []; //logs about the file processing will be stored here

//iteration through each file
async.forEach(file_list , function(file_from_list , callback){
    if(file_is_type1(file_from_list)){
        process_filetype_1(file_from_list);
        callback(null);
    }
    else if(file_is_type1(file_from_list)){
        process_filetype_2(file_from_list);
        callback(null);
    }
    else{
        logs.push("Unknown file type");
    }
},function(error, result){
     return res.status(200).send({message: import_log});
});

There are two different functions with differences in logic that process the files. It is a complicated code, but runs smoothly and each action is performed at the right time. Here is an abstraction of one of the functions:

function process_filetype_1(file_from_list){
    async.auto({
        dosth_1: function(callback) {
            //logic implementation
        },
        dosth_2: ['dosth_1', function(callback) {
            //logic implementation
        }]
    }, function(error, results) {
        return results;
    });
}

What happens: all files are processed, but callbacks inside the iteration don't wait for the function to finish execution. Therefore the response is send before files have finished processing

Why it happens: calling process_filetype_1(file_from_list) and callback(null) is asynchronous. callback(null) finishes first, all files are iterated, the execution flow goes to sending the response. In the meanwhile the functions are still executing.

How to fix it: Haven't figured it out. Ideally, callback would wait for the functions to return their respective values (and finish executing) before returning null.

What cannot be done: using timers to make callback wait. The number of files being processed could be from 1 to hundreds, and there is no limit in size. The amount of time it may take to process each file is unknown.

What I have tried: returning callbacks when the function finishes executing. Assigning the function's return value to a variable (ex. test) and doing callback(test). Neither one worked.

Promises are how I usually deal with this. Not sure if you are using es6 datchley.name/es6-promises — chevybow
– chevybow, Commented Apr 25, 2018 at 18:52

markS · Accepted Answer · 2018-04-25 18:54:44Z

2

you could try something like:

if(file_is_type1(file_from_list)){
   process_filetype_1(file_from_list, callback);
}

and then process_filetype_1 would look something like:

function process_filetype_1(file_from_list, callback){
    async.auto({
        dosth_1: function(cb) {
            //logic implementation
        },
        dosth_2: ['dosth_1', function(cb) {
            //logic implementation
        }]
    }, function(error, results) {
        if(error) { callback (error); 
        } else {
        callback(results); 
        }
    });
}

answered Apr 25, 2018 at 18:54

markS

5805 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

tadman · Accepted Answer · 2018-04-25 18:52:02Z

If you want to use callback-style code then any function which makes calls that take callbacks must itself take a callback function:

function process_filetype_1(file_from_list, cb) {
    async.auto({
        dosth_1: function(callback) {
            //logic implementation
        },
        dosth_2: ['dosth_1', function(callback) {
            //logic implementation
        }]
    }, function(error, results) {
        cb(error, results);
    });
}

Since your function doesn't seem to do anything you can just chain through:

function process_filetype_1(file_from_list, cb) {
    async.auto({
        dosth_1: function(callback) {
            //logic implementation
        },
        dosth_2: ['dosth_1', function(callback) {
            //logic implementation
        }]
    }, cb);
}

An alternative is to use the Promise system possibly in conjunction with async/await to help detangle this code. I've found they're considerably easier to work with, especially with complicated flow control situations.

Collectives™ on Stack Overflow

Make callback wait till function returns result in node js

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related