Write each row of pandas dataframe into a new text file - pythonic way

Question

I was trying to google up if there's a way to parse a pandas dataframe row wise and write the contents of each row into a new text file. My dataframe consists of a single column called Reviews.

I'm looking to do some sentiment analysis on movie reviews and that I need each review to be in a separate text file. Can somebody help me here.

That's going to be very inefficient, what's the purpose for that? — Leb
– Leb, Commented Nov 9, 2015 at 23:23
Just to perform classification. My requirement is in that way — Wolf
– Wolf, Commented Nov 9, 2015 at 23:25
make a file name variable that changes every time you write a new line then open that filename with the w parameter — R Nar
– R Nar, Commented Nov 9, 2015 at 23:32
Can you please suggest the format to write data from dataframe to text file ? @RNar I've been wondering on that for quite a while. Does to_csv work for this ? — Wolf
– Wolf, Commented Nov 9, 2015 at 23:34
i wouldnt suggest it, no. because you want to write a new file for each row, iterate through the rows then just have something like f = open(filename, 'w') then f.write(row) kind of thing. just make sure to change filename each time. — R Nar
– R Nar, Commented Nov 9, 2015 at 23:43

Wolf · Accepted Answer · 2015-11-10 00:28:40Z

9

I've written something like this and it works. anyways thanks for your inputs guys

for index, row in p.iterrows():
    if i > len(p):
       break
    else:
       f = open(str(i)+'.txt', 'w')
       f.write(row[0])
       f.close()
       i+=1

where p is a dataframe.

edited Nov 10, 2015 at 0:28

answered Nov 10, 2015 at 0:23

Wolf

3992 gold badges5 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Laurie Over a year ago

For anyone else receiving the Unicode Error: change f = open(str(i)+'.txt', 'w'), to f = open(str(i)+'.txt', 'w', encoding='utf-8')

Leb · Accepted Answer · 2015-11-10 00:16:57Z

2

It's still inefficient, but since it's required here's one possible solution.

import pandas as pd
from io import StringIO

data="""
column1 column2
c1 c2
c3 c4
c5 c6
"""

df = pd.read_csv(StringIO(data), delimiter='\s+')

i=0
for row in df.values:
    filename = 'testdir/review{}.csv'.format(i)
    row.tofile(filename, sep=",", format="%s")
    i+=1

This will take the values as an array and write the data to a csv file named review0.csv, review1.csv... Another solution is to use pd.to_csv within the loop and specify the chunk

answered Nov 10, 2015 at 0:16

Leb

16k11 gold badges58 silver badges77 bronze badges

Comments

jss367 · Accepted Answer · 2021-03-03 00:35:16Z

1

Here's another way to do it. This creates a destination folder if it doesn't exist.

import pandas as pd
from pathlib import Path

root_location = Path("/my/root/path")
os.makedirs(root_location, exist_ok=True)
df = pd.read_csv(my_csv) # for example

for index, row in df.iterrows():
    with open(root_location / (str(row["file_name"]) + ".txt"), "w") as f:
        f.write(str(row["file_contents"]))

answered Mar 3, 2021 at 0:35

jss367

5,57315 gold badges58 silver badges89 bronze badges

Comments

Quinn_Quinn · Accepted Answer · 2022-06-29 17:46:55Z

0

this is simpler but might be costly solution

for i in range(len(data_to_txt)): 
    data_to_txt.iloc[[i]].to_csv(str(i)+".txt")

edited Jun 29, 2022 at 17:46

answered Jun 29, 2022 at 17:46

Quinn_Quinn

111 silver badge2 bronze badges

Collectives™ on Stack Overflow

Write each row of pandas dataframe into a new text file - pythonic way

4 Answers 4

1 Comment

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related