How to open multiple csv files from a folder in python?

If you a have a directory containing your csv files, and they all have the extension .csv, then you could use, for example, glob and pandas to read them all in and concatenate them into one csv file. For example, say you have a directory, like this:

csvfiles/one.csv
csvfiles/two.csv   

where one.csv contains:

name,age

Keith,23
Jane,25

and two.csv contains:

name,age

Kylie,35
Jake,42

Then you could do the following in Python (you will need to install pandas with, e.g., pip install pandas):

import glob
import os
import pandas as pd

# the path to your csv file directory
mycsvdir = 'csvdir'

# get all the csv files in that directory (assuming they have the extension .csv)
csvfiles = glob.glob(os.path.join(mycsvdir, '*.csv'))

# loop through the files and read them in with pandas
dataframes = []  # a list to hold all the individual pandas DataFrames
for csvfile in csvfiles:
    df = pd.read_csv(csvfile)
    dataframes.append(df)

# concatenate them all together
result = pd.concat(dataframes, ignore_index=True)

# print out to a new csv file
result.to_csv('all.csv')

Note that the output csv file will have an additional column at the front containing the index of the row. To avoid this you could instead use:

result.to_csv('all.csv', index=False)

You can see the documentation for the to_csv() method here.

Hope that helps.

Here is a very simple way to do what you want to do.

import pandas as pd
import glob, os

os.chdir("C:\\your_path\\")
results = pd.DataFrame([])

for counter, file in enumerate(glob.glob("1*")):
    namedf = pd.read_csv(file, skiprows=0, usecols=[1,2,3])
    results = results.append(namedf)

results.to_csv('C:\\your_path\\combinedfile.csv')

Notice this part: glob("1*")

This will look only for files that start with '1' in the name (1, 10, 100, etc). If you want everything, change it to this: glob("*")

Sometimes it's necessary to merge all CSV files into a single CSV file, and sometimes you just want to merge some files that match a certain naming convention. It's nice to have this feature!

I know that the post is a little bit old, but using Glob can be quite expensive in terms of memory if you are trying to read large csv files, because you will store all that data into a list in then you'll still have to have enough memory to concatenate the dataframes inside that list into a dataframe with all the data. Sometimes this is not possible.

dir = 'directory path'
df= pd.DataFrame()

for i in range(0,24):
  csvfile = pd.read_csv(dir+'/file name{i}.csv'.format(i), encoding = 'utf8')
  df = df.append(csvfile)
  del csvfile

So, in case your csv files have the same name and have some kind of number or string that differentiates them, you could just do a for loop trough the files and delete them after they are stored in a dataframe variable using pd.append! In this case all my csv files have the same name except they are numbered in a range that goes from 0 to 23.