How can I parse records on demand with Node.js?

Question

Is there a Node module that can parse a specific number of records from a CSV file? The use case is to parse a large log file and deliver records to a paging client as requested.

node-csv can't yet do this, and the closest I've found is to read lines one by one, which requires reinventing the CSV parsing wheel, and will break on multi-line records.

But let's lower the bar: how can I parse single-line CSV records one by one with Node.js? Pretty trivial task in most other languages.

What do you mean by "single-line CSV records"? Because obviously it's not just 1 line, as my reply got downvoted. — Slavo
– Slavo, Commented Oct 28, 2013 at 9:32
@Slavo: no idea who downvoted your answer (wasn't me). By "single-line CSV records", I mean comma-separated tuples without newlines in any of the fields. As in, the regular type of CSV data. — Dan Dascalescu
– Dan Dascalescu, Commented Oct 28, 2013 at 11:56

Slavo · Accepted Answer · 2013-10-25 13:06:58Z

2

As far as I understand, you just want to parse the comma-separated line of values into an array? If so, try this one:

https://npmjs.org/package/csvrow

answered Oct 25, 2013 at 13:06

Slavo

15.5k11 gold badges49 silver badges60 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

robertklep · Accepted Answer · 2013-10-29 08:35:31Z

1

Parsing a single 'line' (which can also have embedded newlines):

var csv = require('csv'); // node-csv

csv()
  .from.string(SINGLE_LINE_OF_CSV)
  .to.array(function(record) {
    console.log('R', record);
  });

I'm not sure what you mean by 'a specific number of records from a CSV file', or what the issue is exactly. Once you've read the amount you need, just send the response back to the client and you're done.

EDIT: if you want to implement paging, you can use node-csv too:

var csv   = require('csv');
var skip  = 100;
var limit = 10;

csv()
  .from.path('file.csv')
  .on('record', function(row, index) {
    if (index >= skip && index < (skip + limit))
      console.log('R', index);
  });

edited Oct 29, 2013 at 8:35

answered Oct 25, 2013 at 13:06

robertklep

205k38 gold badges421 silver badges412 bronze badges

2 Comments

Dan Dascalescu Over a year ago

I seem to have trouble parsing the sample code. How is "SINGLE_LINE_OF_CSV" read, such that it has embedded newlines? And how is the csv function defined? By "specific number of records", I mean reading synchronously N records from a CSV file, instead of parsing the entire file in memory and then picking the desired records.

robertklep Over a year ago

@DanDascalescu sorry, you lowered the bar :) The example code parses single-line CSV records (and you can call it multiple times to achieve the 'one-by-one' aspect). You didn't mention where they should come from. And csv is node-csv. Perhaps you should just make your question more clear as everyone seems to be failing to implement your intention...

Collectives™ on Stack Overflow

How can I parse records on demand with Node.js?

2 Answers 2

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related