Save pandas groups to separate CSV files

Question

Is it possible to return multiple df's to independent csv's without using .to_csv. Using the code below I'm manually returning desired values and exporting those to a csv. This is ok if I only have a few files to export but if there's numerous files or file names that constantly change over datasets it can be cumbersome.

Is there a more efficient way to return desired values if you have a specific value list and export those to a csv?

import pandas as pd

d = ({
    'C' : ['08:00:00','XX','08:10:00','XX','08:41:42','XX','08:50:00','XX', '09:00:00', 'XX','09:15:00','XX','09:21:00','XX','09:30:00','XX','09:40:00','XX'],
    'D' : ['Home','Home','Home','Home','Away','Away','Shops','Shops','Away','Away','Shops','Shops','Home','Home','Away','Away','Home','Home'],
    'E' : ['Num:','','Num:','','Num:','','Num:','','Num:', '','Num:','','Num:','','Num:', '','Num:', ''],
    'F' : ['1','','1','','1','','1','','1', '','2','','2','','1', '','2',''],   
    'A' : ['A','','A','','A','','A','','A','','A','','A','','A','','A',''],           
    'B' : ['Stop','','Res','','Stop','','Start','','Res','','Stop','','Res','','Start','','Start','']
    })

df = pd.DataFrame(data=d)

#List of designated places
values = ['Home', 'Away', 'Shops']

#Export to csv
Home = df.loc[df['D'] == 'Home'].to_csv('Home.csv')
Away = df.loc[df['D'] == 'Away'].to_csv('Away.csv')
Shops = df.loc[df['D'] == 'Shops'].to_csv('Shops.csv')

Sorry, fixed the duplicate link. You may want something like this: stackoverflow.com/questions/45871314/… — cs95
– cs95, Commented Jun 13, 2018 at 4:45

cs95 · Accepted Answer · 2019-04-14 18:24:00Z

5

Filter with isin, then perform a groupby on "D" and iteratively save to CSV.

incl = ['Home', 'Away', 'Shops']    
for k, g in df[df['D'].isin(incl)].groupby('D'):
    g.to_csv(f'{k}.csv')  # '{}.csv'.format(k)

The isin filtering step is important if and only if you have more categories than what you want to save. If this isn't the case and you want to save everything, your solution will simplify:

for k, g in df.groupby('D'):
    ...

edited Apr 14, 2019 at 18:24

answered Jun 13, 2018 at 4:46

cs95

406k106 gold badges744 silver badges797 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

user9394674 Over a year ago

Thanks @coldspeed

user9394674 Over a year ago

This will probably have to be another question but could I add new columns before exporting?

cs95 Over a year ago

@PeterJames123 it depends on what the columns are you want to add. Anything you need to do can be done inside the for loop.

user9394674 Over a year ago

Thanks @coldspeed. I got it half working stackoverflow.com/q/50831061/9394674

Collectives™ on Stack Overflow

Save pandas groups to separate CSV files

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related