Convert bytes csv string into pandas dataframe

Question

I am pulling in a csv string and need to parse it into a pd dataframe. Example string:

b'date,"total revenue"\n2018-06-19,12.65\n2018-06-20,3.90\n2018-06-21,6.16\n2018-06-22,9.06\n2018-06-23,1.30\n2018-06-24,1.88\n2018-06-25,4.20\n2018-06-26,2.46\n2018-06-27,2.38\n2018-06-28,1.06\n`

How can I convert this string into a 2 column pandas dataframe? Many thanks

I tried the following:

df = pd.DataFrame(list(reader(str(my_string))))

But the output is all skewed:

                   0     1
0                  b  None
1                  '  None
2                  d  None
3                  a  None
4                  t  None
5                  e  None
6                         
7      total revenue  None
8                  \  None

Laurence_jj · Accepted Answer · 2020-02-26 11:41:03Z

3

I found the answer here (How to convert bytes data into a python pandas dataframe?):

from io import StringIO

s=str(bytes_data,'utf-8')

data = StringIO(s) 

df=pd.read_csv(data)

answered Feb 26, 2020 at 11:41

Laurence_jj

7681 gold badge11 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Durand · Accepted Answer · 2022-08-04 14:09:56Z

3

You can simply use io.BytesIO instead of converting to string and using io.StringIO :
For example, retrieving CSV data from an API :

import io
import requests
r = requests.get("http://127.0.0.1:5000/get_logs")
data = io.BytesIO(r.content)
df = pd.read_csv(data)

(This is what Eric proposed above, but without the need to actually save the data to a persistent static file)

answered Aug 4, 2022 at 14:09

Durand

891 silver badge9 bronze badges

Comments

Manish Chaudhary · Accepted Answer · 2020-02-26 12:07:56Z

1

import sys
import pandas as pd

if sys.version_info[0] < 3: 
    from StringIO import StringIO
else:
    from io import StringIO


temp =  StringIO(my_string)
df =pd.read_csv(temp, sep=",", lineterminator='\n')

df

You migth try this, and dont forget to remove b as it take rows for df as binary

answered Feb 26, 2020 at 12:07

Manish Chaudhary

5186 silver badges15 bronze badges

Comments

ERIC · Accepted Answer · 2021-07-16 03:26:37Z

1

I use this way if the binary data is a csv file.

with open("test.csv", 'wb') as f:  
    f.write(bytes_data)
df = pd.read_csv('test.csv')

answered Jul 16, 2021 at 3:26

ERIC

5006 silver badges17 bronze badges

Collectives™ on Stack Overflow

Convert bytes csv string into pandas dataframe

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related