How Merge multi dataframes Pandas in Python

Question

FinalDf = pd.merge(HRTrainingData,HRDataSet1,HRDataSet2,HRDataSet3, on='EmployeeNumber', how='outer')

TypeError: merge() got multiple values for argument 'on'

I have only entered one 'on' argument, so I'm not sure what is going on here, but I am unable to merge these data frames. Any advice?

I believe your problem is because you are trying to merge on more than 2 dataframes at once. You might need to either use concat or just do a different merge for each dataframe — ArchAngelPwn
– ArchAngelPwn, Commented Jul 12, 2022 at 18:42
@ArchAngelPwn I actually have 15 data frames to merge into one... I tried to start with less because I cant figure out how to do 15. — Brandi Austin
– Brandi Austin, Commented Jul 12, 2022 at 18:44
@ArchAngelPwn I tried concat but my number of rows don't match my csv file because I cant merge them on a specific column. — Brandi Austin
– Brandi Austin, Commented Jul 12, 2022 at 18:44

Mahdi F. · Accepted Answer · 2022-07-12 18:50:01Z

3

You can write like below:

pd.merge(df3, pd.merge(df1,df2, on='EmployeeNumber', how='outer'), on='EmployeeNumber',how='outer')

Or with functools.reduce:

import functools

functools.reduce(lambda x,y : pd.merge(x,y, 
                                       on='EmployeeNumber', 
                                       how='outer'), 
                 [df1, df2, df3, df4, ..., df15])
# above code inference like below
# merge(merge(merge(merge(df1, df2), df3), df4), ..., df15)

edited Jul 12, 2022 at 18:50

answered Jul 12, 2022 at 18:46

Mahdi F.

24.1k5 gold badges25 silver badges32 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Brandi Austin Over a year ago

Thank you, I will give that a go. I actually have 15 data frames to merge in total.

Mahdi F. Over a year ago

@BrandiAustin, you can use second part for 15 dataframe

Brandi Austin Over a year ago

ValueError: You are trying to merge on object and int64 columns. If you wish to proceed you should use pd.concat

Mahdi F. Over a year ago

@BrandiAustin, convert all columns EmployeeNumber in all dataframes to int or to str then merge them

Scott Boston · Accepted Answer · 2022-07-12 18:51:53Z

1

Or you can use join,

df_list=[d.set_index('EmployeeNumber', inplace=True) for d in [df1,df2,d3,d4]]

df_list[0].join(df_list[1:])

One of the advantages pd.DataFrame.join over pd.DataFrame.merge, join accepts a list of dataframes.

answered Jul 12, 2022 at 18:51

Scott Boston

154k15 gold badges160 silver badges207 bronze badges

2 Comments

Brandi Austin Over a year ago

KeyError: "None of ['EmployeeNumber'] are in the columns"

Scott Boston Over a year ago

If employeeNumber is already in the index then remove set_index part and just create a list of your dataframes.

Collectives™ on Stack Overflow

How Merge multi dataframes Pandas in Python

2 Answers 2

4 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related