How to properly convert an objects array to a CSV file with Node.js?

Question

I need to perform a mongo query and convert results to CSV file. I've came across with node-csv but I can't figure out how to pass the query results to this module to convert it and actually generate a file.

So I've read this (adapting my code a bit to fit my query example):

var generate = require('csv-generate');

var generator = generate({objectMode: true, headers: 7});

var columns = {
  _id: '_id',
  first_name: 'first_name',
  last_name: 'last_name',
  team: 'team.name',
  position: 'position',
  status: 'status'
};
var stringifier = stringify({ header: true, columns: columns });

generator.pipe(stringifier).pipe(process.stdout);

And I do queries like:

Player.find({},{_id: true, first_name: true, last_name: true, "team.name": true, position: true, status:true}).exec(function (err,players){
  // Should CONVERT to CSV string AND generate file HERE
});

However I really can't figure out what steps I should follow in order to generate a CSV string and convert it to file from the example.

Ionică Bizău · Accepted Answer · 2015-07-16 19:33:47Z

1

The good way (which is also memory friendly) is to use streams. First, you do the MongoDB request:

collection.find({ some: "thing" }, function (err, cursor) {
    if (err) { return ...; }
    function next () {
      cursor.nextObject(function (err, obj) {
         if (err) { return ...; }
         if (obj === null) {
              return outputStream.end();
         }
         outputStream.write(stringify(obj));
         next();
      });
    }
    next();
});

...the outputStream could be an HTTP response stream (supposing you are inside of a http server request - response) or any other writable stream.

This is more or less just the basic example -- probably it could be improved, but that's how I recommend it: stringify objects one by one and write them in the stream.

answered Jul 16, 2015 at 19:33

Ionică Bizău

114k95 gold badges311 silver badges490 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

hassansin · Accepted Answer · 2015-07-16 17:31:46Z

1

With mongoose, you can get a readable stream with stream() method. Then just pipe this with csv transform stream:

  var stringifier = csv.stringify({ header: true, columns: columns });
  Player.find()
  .stream()
  .pipe(csv.transform(function(record){
    record = record.toObject() //convert mongoose document to JS object
    return Object.keys(record).map(function(key){return record[key]}) // return array of columns
  }))
  .pipe(stringifier)
  .pipe(fs.createWriteStream('out.csv'));

edited Jul 16, 2015 at 17:31

answered Jul 16, 2015 at 17:22

hassansin

17.6k3 gold badges47 silver badges49 bronze badges

Comments

David Worms · Accepted Answer · 2015-07-16 18:01:53Z

1

You could also simply return the js object from the handler function in "csv.transform". "csv.stringify" will accept objects when the columns options is provided.

answered Jul 16, 2015 at 18:01

David Worms

111 bronze badge

1 Comment

diegoaguilar Over a year ago

Could you extend more in your answer?

Collectives™ on Stack Overflow

How to properly convert an objects array to a CSV file with Node.js?

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related