I'm trying to insert some data into my MongoDB collection using async's waterfall method, to make sure that everything is done in the correct order. Initially my data was being inserted into MongoDB in the wrong order, which after a while I have realised is because of the fact my insert was running asynchronously so there was no way for me to guarantee that the data would be saved in the correct order.
So here is my async.waterfall:
async.waterfall([
function(callback) {
request.get(apiUrl, function(error, response, body) {
if (!error && response.statusCode == 200) {
data = JSON.parse(body);
callback(error, data.items);
}
});
},
function(data, callback) {
removeInvalidItems(data);
var items = convertToObj(data);
callback(null, items);
},
function(items, callback) {
console.log(items);
// Insert to collection here
}
], function (err, result) {
// do something here
});
So on the last function in the waterfall I have logged items to console, and it is showing the data in the correct order. So now, all I need to do is insert the data to my collection, and in the callback close the connection and move out of the waterfall to the final function. Easy enough right? Well, this is what I am trying right now:
MongoClient.connect(config.db, function(err, db) {
if (err) {
console.log(err);
} else {
console.log("Connected correctly to server");
db.collection('items').insert(items, function(err, result) {
if (err) {
console.log(err);
}
db.close();
callback(null, 'done');
});
}
});
The db.close() and callback(null, 'done') should be closing the connection and moving out of the waterfall once they receive the callback. However all I seem to get is RangeError: Maximum call stack size exceeded. I have tried everything that I can think of and I can't get this insert to work with a callback and move out of the waterfall.
Can anyone help?
itemsvariable a plain JSON object?iteminitemsis stored as a newItemobject according to a mongoose Schema. I believe I am doing something wrong with the callback, everything works up until the insert.