How do I add a header row to a CSV file using node-csv?

Question

I am parsing a bunch of csv files with node using node-csv. I have hundreds of files that need to be parsed, however, I need to add a header row to each file to make use of the 'columns' option in the parser. The columns option parses each row as an object with the header row serving as object keys, however for this option to work, you of course need a header row.

By looking at the docs, my gut tells me I should be able to pipe my node stream through a transform function that adds the row, but I'm having a difficult time making this work without altering the existing data in the file.

Here is what I'm thinking, but how do I write a row to the 'zero' column?

let createStream = function() {
   let stream = fs.createReadStream(transPath+'/'+file, {encoding: 'utf8'});
   let parser = csv.parse({columns: true});
   let transform = csv.transform(function(record, doneTransform){
                             //check if first zero row,
                             //Add header column. 
                             doneTransform(); 
                  }); 
   return stream.pipe(transform).pipe(parser);
};

    createStream().on('data', function(transaction){
        //do stuff with object data
    });

idbehold · Accepted Answer · 2016-10-14 20:34:00Z

4

The real solution:

let createStream = function() {
  let stream = fs.createReadStream(transPath+'/'+file, {encoding: 'utf8'});
  let parser = csv.parse({skip_empty_lines: false, auto_parse: true, columns: header});
  return stream.pipe(parser);
};

createStream().on('data', function(transaction){
        q.push(transaction);
});

answered Oct 14, 2016 at 20:34

idbehold

17.2k5 gold badges49 silver badges75 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

calbear47 Over a year ago

Thanks for adding this solution. I agree with you, this is a better solution.

calbear47 · Accepted Answer · 2016-10-14 20:22:59Z

I solved this problem by looking at the issue a little differently. Let me explain.

First, My initial attempt above didn't work because it used the Transform and Parse packages incorrectly. You can use them independently, as the package suggests, but if you do decide to use them together, the proper usage requires that you parse your CSV first, and then transform it...Thus, my attempt above is a dead on arrival.

Secondly, once I backed away from my requirement to use headers, I realized that what I really wanted was an object with the correct key/value pairs, which led me to trying to transform the data on my own, instead of relying upon the 'columns' option to do this for me.

This led to my result: If i can guarantee the order, and number of columns for each record, then can build an object from two arrays with the corresponding data in a transform function.

Here is the code:

let createStream = function() {
  let stream = fs.createReadStream(transPath+'/'+file, {encoding: 'utf8'});
  let parser = csv.parse({skip_empty_lines: false, auto_parse: true});
  let transform = csv.transform(function(record, doneTransform){
                   let newObject = _.zipObject(header, record);
                   doneTransform(null, newObject);
                  });
  return stream.pipe(parser).pipe(transform);
};

createStream().on('data', function(transaction){
        q.push(transaction);
});

'header' is an array of keys I wish to pair up with a corresponding value that I parse from the CSV.

I am making use of lodash's zipObject function that creates an Object from two arrays. You can find that here: zipObject.

Hope this helps someone find their solution when they don't have headers on their CSV files.

Collectives™ on Stack Overflow

How do I add a header row to a CSV file using node-csv?

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related