Node.js - using request to send multiple HTTP GET's in a for loop

Question

I'm trying to send multiple HTTP requests from my node.js server (using the 'request' package) in a for loop.

The idea is to basically have an array of URLs, send a HTTP get for each one, and store the response in an array called 'responseArray'.

When I try this, I get 'undefined', but I know the requests are working if I log them to the console inside the for loop.

function apiCalls() {

  var URLs = [‘URL1’, ‘URL2’, ‘URL3’];
  var responseArray = []
  for (var i = 0; i < URLs.length; i++) {
    request({
      url: URLs[i],
      method: 'GET',
      headers: {
        'Connection': 'close'
      },
    }, function(error, response, body) {
      if (error) {
        console.log(error);
      } else {
        responseArray.push(String(response.body));
        console.log(responseArray) // this is showing me all the data in the array correctly
        return responseArray;
      }
    }) //end request function
  } //end for loop
  console.log(responseArray)
} //end apiCalls function
apiCalls()

So after looking at a few different solutions here on stack overflow and elsehwere I tried using promises. I've never used them before and based this off the google example

Promise:

var apiCalls = new Promise(function(resolve, reject) {
  var URLs = [‘URL1’, ‘URL2’, ‘URL3’];
  var responseArray = [];
  for (var i = 0; i < URLs.length; i++) {
    process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";
    request({
      url: URLs[i],
      method: 'GET',
      headers: {
        'Connection': 'close'
      },
    }, function(error, response, body) {
      if (error) {
        console.log(error);
      } else {
        resolve(responseArray.push(String(response.body))
      }
    }) //end request

  } //end for loop
})

apiCalls.then(function(result) {
  console.log('this is calling the promise')
  console.log(result)
}, function(err) {
  console.log(err)
});

I always get an empty array when I try to log the responseArray after the for loop. Alternatively - I get 'undefined' if I try to assign the returned array to a variable like so:

var gimmeAllResponses = apiCalls();
console.log(gimmeAllResponses); //returns 'undefined'

Can anyone show me where I'm going wrong? how do I updated the 'responseArray' data after the for loop has finished?

Where are you getting undefined? On the console? Is it the console.log(error) statement, you think? — Armen Michaeli
– Armen Michaeli, Commented Apr 17, 2017 at 11:18

Tatsuyuki Ishi · Accepted Answer · 2017-04-17 11:24:38Z

1

This is a little bit off, since this requires the alternative package, request-promise.

You're resolving many times. Since you're using Node.js, it's very likely that ES6 features are available. Use Array.prototype.map() and Promise.all():

var rp = require('request-promise');
var URLs = [‘URL1’, ‘URL2’, ‘URL3’];
process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";
var responseArray = Promise.all(URLs.map((url) => rp({
  uri: url,
  method: 'GET',
  headers: {
    'Connection': 'close'
  }
}).then((error, response, body) => {
    if (error) {
      console.log(error);
    } else {
      return String(response.body);
    }
  })));

answered Apr 17, 2017 at 11:24

Tatsuyuki Ishi

4,0714 gold badges31 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Node.js - using request to send multiple HTTP GET's in a for loop

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related