Merge a list of dataframes to create one dataframe [duplicate]

Question

I have a list of 18 data frames:

dfList = [df1, df2, df3, df4, df5, df6.....df18]

All of the data frames have a common id column so it's easy to join them each together with pd.merge 2 at a time. Is there a way to join them all at once so that dfList comes back as a single dataframe?

could you elaborate on the looping option? sounds like what I'd be looking for — Josh
– Josh, Commented Aug 16, 2016 at 15:02

Jinhua Wang · Accepted Answer · 2020-08-04 21:17:24Z

119

I think you need concat, but first set index of each DataFrame by common column:

dfs = [df.set_index('id') for df in dfList]
print pd.concat(dfs, axis=1)

If need join by merge:

from functools import reduce
df = reduce(lambda df1,df2: pd.merge(df1,df2,on='id'), dfList)

edited Aug 4, 2020 at 21:17

Jinhua Wang

1,7791 gold badge19 silver badges46 bronze badges

answered Aug 16, 2016 at 14:57

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Josh Over a year ago

pd.concat(dfList, axis=1) combined them all, but each df had it's own line (so every id was repeated 18 times)

Peter.k Over a year ago

I tried to write as variable: sdf = pd.concat(dfs, axis=1) and got TypeError: first argument must be an iterable of pandas objects, you passed an object of type "DataFrame" Don't get it.

Collectives™ on Stack Overflow

Merge a list of dataframes to create one dataframe [duplicate]

1 Answer 1

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Linked

Related