Writing to io.BytesIO in csv fails in python3

Question

I am trying to write python 2/3 compatible code to write strings to csv file object. This code:

line_as_list = [line.encode() for line in line_as_list]
writer_file =  io.BytesIO()
writer = csv.writer(writer_file, dialect=dialect, delimiter=self.delimiter)
for line in line_as_list:
    assert isinstance(line,bytes)
    writer.writerow(line)

Gives this error on Python3:

>           writer.writerow(line)
E           TypeError: a bytes-like object is required, not 'str'

But assert has no problem with the type, so why is csv creating an error?

Can't I use BytesIO only for both Python 2 and 3? Where is the problem here?

@tdelaney What I meant was I am not sure whether StringIO and BytesIO will give the same representation for source text (probably in utf-8). Thats why I am trying to use the same output object type. — goelakash
– goelakash, Commented Jun 22, 2016 at 17:10

unutbu · Accepted Answer · 2016-06-22 17:44:39Z

41

In Python3 csv.writer expects a file-like object opened in text mode. In Python2, csv.writer expects a file-like object opened in binary mode.

Therefore, in Python3, use io.StringIO, while in Python2 use io.BytesIO:

import io
import csv
import sys
PY3 = sys.version_info[0] == 3

line_as_list = [u'foo', u'bar']
encoding = 'utf-8'

if PY3:
    writer_file =  io.StringIO()
else:
    writer_file =  io.BytesIO()
    line_as_list = [line.encode(encoding) for line in line_as_list]

writer = csv.writer(writer_file, dialect='excel', delimiter=',')
writer.writerow(line_as_list)
content = writer_file.getvalue()

if PY3:
    content = content.encode(encoding)

print(type(content))
print(repr(content))

In Python3 the code above prints

<class 'bytes'>
b'foo,bar\r\n'

In Python2 the code above prints

<type 'str'>
'foo,bar\r\n'

edited Jun 22, 2016 at 17:44

answered Jun 22, 2016 at 17:34

unutbu

886k197 gold badges1.9k silver badges1.7k bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

goelakash Over a year ago

That's a good workaround, but any idea why the error asks for 'bytes', when str is a byte format?

unutbu Over a year ago

I believe that error is coming from the BytesIO object -- it is complaining that it was passed a str when it expected bytes. In Python3 a str is not a "byte format". A unicode str is a sequence of code points.

goelakash Over a year ago

But I passed a str.encode() object, effectively a bytes object. Then where is the problem? This error says that str was passed, when it wasn't (just talking about Python 3).

unutbu Over a year ago

I'm not able to reproduce the error you posted so this is just a guess. What is self.delimiter? Could it have been a str?

goelakash Over a year ago

Yeah, that may be it, though after encoding the delimiter it says that 'the delimiter must be string, not bytes'.

|

Collectives™ on Stack Overflow

Writing to io.BytesIO in csv fails in python3

1 Answer 1

7 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related