Async.js design pattern

Question

I am using async.js for my node.js app. I need help solving the following problem.

Let's say I have a following aysnc js series function.

async.waterfall([
    function getDataFromDB(request, response, callback){ ... },
    function someOperationOnDBData(dbData, response, callback){ ... },
    function renderSomeFlow(evaluatedData, response, callback){ ... }
]);

I have three functions called in the order mentioned above. I am getting data from getDataFromDB and passing to someOperationOnDBData and so on.

Suppose I need one more operation in between getDataFromDB and someOperationOnDBData but still pass DBData forward. Eg:

async.waterfall([
    function getDataFromDB(request, response, callback){ ... },
    function extraOperation(dbData, response, callback) {...}
    function someOperationOnDBData(dbData, extraOperationData, response, callback){ ... },
    function renderSomeFlow(evaluatedData, response, callback){ ... }
]);

Here adding a single step in the middle changes the function definitions and also I need to pass dbData in extraOperation to just forward it to someOperationOnDBData. Also, if I am calling a different module in the middle, it might not be possible to change it's parameter to forward some data.

How to solve this problem of passing data between functions in async.js without forwarding data in middle functions? Refactoring functions every time a new step is included is not possible. What is the design pattern for solving this type of problem?

@Hitmands could you please elaborate more? how will Promises solve this problem? — dejavu
– dejavu, Commented Jun 22, 2016 at 10:12
@dejavu Promises would require refactoring of your methods as you can't waterfall with native Promises. What you can do however is send a single object through the chain which you manipulate and add to. Then you don't change function signatures in every method in that chain. So long as you don't have conflicting property names that is. Plus it makes complex flow far easier than callbacks and flattens out the chain. No more callback hell. I would choose promises over callbacks every time. — ste2425
– ste2425, Commented Jun 22, 2016 at 10:43
@ste2425 is passing a single object and manipulating/adding new value to it regarded as good practice? I don't have much experience in node but I am not entirely convinced by it. Could you point to some blog post etc to read more about it? — dejavu
– dejavu, Commented Jun 22, 2016 at 12:56
About NodeJS or ES6 Promises? Google and SO will throw up tons of good info as to why they both are brilliant. Obviously if your from a .NET background NodeJS can be unpleasant as most JavaScript is for someone like that. — ste2425
– ste2425, Commented Jun 22, 2016 at 13:09

MrWillihog · Accepted Answer · 2016-06-22 10:07:58Z

4

Using waterfall you have to pass the data through - you can't stop a waterfall in the middle :). There are alternatives that you might like to use, for example, auto which lets you specify which functions depend on the results of other functions and async will determine the best order to run them.

I find the syntax of auto a little awkward but it does what you need. Here is an example:

async.auto({
    db: function getDataFromDB(callback){ ... },
    extra: ['db', function extraOperation(results, callback) {...}],
    some: ['db', function someOperationOnDBData(results, callback){ ... }],
    render: ['some', 'db', function renderSomeFlow(results, callback){ ... }]
});

edited Jun 22, 2016 at 10:07

answered Jun 22, 2016 at 9:57

MrWillihog

2,67620 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

YeeHaw1234 · Accepted Answer · 2017-11-23 14:16:37Z

0

I've found that currying your functions with arrow functions is a way arround this problem:

const someOperationOnDBData = extraOperationData => (dbData, response, callback) => {
  // do your function work
};

async.waterfall([
  function getDataFromDB(request, response, callback){ ... },
  function extraOperation(dbData, response, callback) {...}
  someOperationOnDBData(extraOperationData)(dbData, response, callback),
  function renderSomeFlow(evaluatedData, response, callback){ ... }
]);

Many good examples on how to curry javascript functions on Medium.com

answered Nov 23, 2017 at 14:16

YeeHaw1234

2,4951 gold badge18 silver badges17 bronze badges

Collectives™ on Stack Overflow

Async.js design pattern

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related