I currently use the following code to create a binary file that I then directly upload into AWS S3. Now I was told it's possible to write with the csv.writer directly into the binary mode and avoid the extra step with io.StringIO(). How does that work?
buffer = io.StringIO()
writer = csv.writer(buffer)
writer.writerow(["a", "b", "c"])
buffer_2 = io.BytesIO(buffer.getvalue().encode())
BUCKET_NAME = 'fbprophet'
OBJECT_NAME = 'blah.csv'
s3.upload_fileobj(buffer_2, BUCKET_NAME, OBJECT_NAME)
What you've got there looks reasonable to me. The post you link to talks about writing to files, not in-memory streams. A file can be opened in either text or binary mode, which determines whether it operates on strings (str) or raw bytes (bytes). But the in-memory file-like objects from the io package aren't as flexible: you have StringIO for strings, and BytesIO for bytes.
Because csv requires a text stream (strings), and boto requires a binary stream (bytes), a conversion step is necessary.
I would recommend to pass the actual encoding to the encode() function though, to avoid falling back to Python's system-dependent default:
buffer_2 = io.BytesIO(buff.getvalue().encode('utf-8'))
Python 3openisn't going to cut it. I had an answer already written, the short of it is: your approach is fine, I don't see a way to significantly improve it. Just be sure to passencoding('utf-8')because Python's default encoding is system-depedent.