line contains NULL byte error in python csv reader

Question

I am trying to read each line of a csv file and get a "line contains NULL byte" error.

reader = csv.reader(open(mycsv, 'rU'))
for line in reader:
     print(line)


Traceback (most recent call last):
  File "<stdin>", line 1, in <module
_csv.Error: line contains NULL byte

Using the below I found that I have null bytes.

if '\0' in open(mycsv).read():
     print("have null byte")

What's the best way to work around this? Do a replace '\0' on all lines? I need to process this kind of file daily and have about 400,000 lines (1Gb) of data. I assume a replace would substantially slow this down even more.

hjpotter92 · Accepted Answer · 2017-05-14 17:41:07Z

11

Try this!

import csv 

def mycsv_reader(csv_reader): 
  while True: 
    try: 
      yield next(csv_reader) 
    except csv.Error: 
      # error handling what you want.
      pass
    continue 
  return

if __name__ == '__main__': 
    reader = mycsv_reader(csv.reader(open(mycsv, 'rU')))
    for line in reader:
        print(line)

edited May 14, 2017 at 17:41

hjpotter92

81.1k36 gold badges148 silver badges188 bronze badges

answered Sep 26, 2014 at 2:33

han058

9088 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Eric Over a year ago

That works and gets me through the file, I am just wondering why I am getting these null bytes. Are they maybe used instead of commas as separators? f.count('\x00') returns 1926 of these.

han058 Over a year ago

Please refer to the stackoverflow.com/questions/7894856/…

Collectives™ on Stack Overflow

line contains NULL byte error in python csv reader

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related