How to transform synchronous style code to asynchronous code?

Question

I have some synchronous code that looks like this:

function bulk_upload(files, callback) { 
  for (file in files) {
   sync_upload(file); // blocks till file uploads
  }
  print('Done uploading files');
  callback();
}

I now have to use an asynchronous API async_upload(file, callback) instead of sync_upload(file) to do the same. I have various options but not sure what is the best:

1) Use a sleep after the for-loop - that's a hack as I have to hope my timing is correct

2) Recursively chain my array:

function bulk_upload(files, callback) {
  if (files.length == 0) {
    print('Done uploading files');
    callback();
  } else {
    async_upload(files.removeLast(), function() { bulk_upload(files, callback); });
  }
}

This is not only hacky but sub-optimal as I could have uploaded my files in parallel using the new async_upload API but I ended up uploading sequentially.

3) Use a global counter:

function bulk_upload(files, callback) {
  uploads = 0
  for(file in files) {
    async_upload(file, function() { uploads++; }); 
  }
  while(uploads < files.length) ; // Empty spin - this is stupid
  print('Done uploading files');
  callback();
}

4) Slightly better counter (but still awful):

function bulk_upload(files, callback) {
  uploads = 0
  for(file in files) {
    async_upload(file, function() { 
      if (++uploads == files.length) { // this becomes uglier as I want to await on more stuff
        print('Done uploading files');
        callback();
      }; 
    }); 
  }
}

You can use futures to get this to work. There's a nice example [here][1]. [1]: stackoverflow.com/questions/5989538/… — Michael Anderson
– Michael Anderson, Commented Oct 1, 2012 at 0:27
The futures answer is the cleanest way, and your 4) is the other mroe standard way, though often wrapped in a helper. I just wanted to point out that your 1) and 3) won't work anyway. Your spinlock will never finish because Javascript uses a single thread, so while your while is looping, it is impossible for the upload callback to run. — loganfsmyth
– loganfsmyth, Commented Oct 1, 2012 at 1:38
Note that option 3) isn't viable; blocking in a tight loop will prevent any of async_uploads callbacks from running so you'll loop infinitely. — ebohlman
– ebohlman, Commented Oct 1, 2012 at 1:38

JohnnyHK · Accepted Answer · 2012-10-01 02:06:15Z

3

You can use the async module's forEach method to do this:

function bulk_upload(files, callback) {
    async.forEach(files, async_upload(file, callback), function (err) {
        if (err) {
            console.error('Failed: %s', err);
        } else {
            console.log('Done uploading files');
        }
        callback(err);
    });
}

answered Oct 1, 2012 at 2:06

JohnnyHK

313k70 gold badges632 silver badges479 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

loganfsmyth · Accepted Answer · 2012-10-01 01:34:34Z

2

Further to my comment, it looks like this code will suffice using futures (untested).

function aync_upload_promise(file) {
    // create a promise.
    var promise = Futures.promise();
    async_upload( file, function(err, data) {
        if (err) {
            // break it
            promise.smash(err);
        } else {
            // fulfill it
            promise.fulfill(data);
        }
    });
    return promise;
}

var promises = [];
for(var i=0; i<files.length; ++i )
{
   promises.push( aync_upload_promise( files[i] ) );
}

Futures
   .join( promises )
   .when( function() {
      print('Done uploading files');
      callback();
   } )
   .fail( function(err) { print('Failed :(', err); } )

edited Oct 1, 2012 at 1:34

loganfsmyth

162k31 gold badges349 silver badges259 bronze badges

answered Oct 1, 2012 at 0:39

Michael Anderson

74.3k9 gold badges147 silver badges196 bronze badges

Collectives™ on Stack Overflow

How to transform synchronous style code to asynchronous code?

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related