Python: Multiple CSVs

Question

I am trying to read a number of CSV files into arrays like

for files in folder:
    with open(files) as f:
         df = pd.read_csv(f)
         result = function(df)
         print result

The problem is this only reads a file at a time not multiple files. My aim is to pass each of these dataframes through a function and create an output for each dataframe. The function is done, all I need to do is read these files into separate dataframes. Is there a method to doing this?

You need to read file by file. You can't read all files at the same time - without threading. — furas
– furas, Commented Jun 18, 2014 at 11:41
@furas could I define dataframes like df1,...dfN using a for loop? — Black
– Black, Commented Jun 18, 2014 at 11:45
You can have list of dataframes - df[0], df[1], .. df[N-1] — furas
– furas, Commented Jun 18, 2014 at 11:46
@furas is it possible? something like for i in range(len(folder)): df_%i = pd.DataFrame() % i so I can then save my CSVs or output in these dataframes — Black
– Black, Commented Jun 18, 2014 at 11:48

furas · Accepted Answer · 2014-06-18 12:09:49Z

1

Using list for dataframes

df = []

for one_file in folder:
        with open(one_file) as f:
            df.append( pd.read_csv(f) )

print df[0]
print df[1] 
# etc.

or

df = []
result = []

for one_file in folder:
        with open(one_file) as f:
            df.append( pd.read_csv(f) )
            result.append( function(df[-1]) )
            print result[-1]

print df[0]
print df[1] 
# etc.
print df[-1] # last df

print result[0]
print result[1] 
# etc.
print result[-1] # last result

edited Jun 18, 2014 at 12:09

answered Jun 18, 2014 at 11:44

furas

149k12 gold badges121 silver badges171 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Black Over a year ago

What does the one_file in files do? Why can't i just use files in folder? Doesn't this just end up reading the letters in the filenames?

furas Over a year ago

I thought files is list of files.

Black Over a year ago

ah sorry. Should have made that clear. folder = os.listdir('path') Made slight alterations in your code and it works fine. Thanks for the help.

furas Over a year ago

I think there is no special function for filename but python is very elastic - try df.filename = "hello_world.cvs"' :)

Collectives™ on Stack Overflow

Python: Multiple CSVs

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related