There are a number of ways to approach this type of problem. Firstly, if you can run all the API calls in parallel (all in flight at the same time) and it doesn't matter what order they are inserted in your database, then you can get a result a lot faster by doing that (vs. serializing them in order).
In all the options below, you would use this code:
const rp = require('request-promise');
function apicall(urlApi){
return rp({url: urlApi, json: true}).then(function(obj){
return dbInsert(obj);
});
}
function dbInsert(obj) {
//insert into database
// return a promise that resolves when the database insertion is done
}
Parallel Using ES6 Standard Promises
let promises = [];
for (let i = 0; i <= data.length; i++) {
promises.push(apicall(data[i]));
}
Promise.all(promises).then(() => {
// all done here
}).catch(err => {
// error here
});
Parallel using Bluebird Promise Library
With the Bluebird Promise library, you can use Promise.map() to iterate your array and you can pass it the concurrency option to control how many async calls are in flight at the same time which might keep from overwhelming either the database or the target API host and might help control max memory usage.
Promise.map(data, apiCall, {concurrency: 10}).then(() => {
// all done here
}).catch(err => {
// error here
});
In Series using Standard ES6 Promises
If you have to serialize them for some reason such as inserting into the database in order, then you can do that like this. The .reduce() pattern shown below is a classic way to serialize promise operations on an array using standard ES6:
data.reduce(data, (p, item) => {
return p.then(() => {
return apicall(item);
});
}, Promise.resolve()).then(() => {
// all done here
}).catch(err => {
// error here
});
In Series Using Bluebird Promises
Bluebird has a Promise.mapSeries() that iterates an array in series, calling a function that returns a promise on each item in the array which is a little simpler than doing it manually.
Promise.mapSeries(data, apiCall).then(() => {
// all done here
}).catch(err => {
// error here
});
iincrementing shouldn't be causing a problem in your case. Your problem is likely something else.