I have two columns,
A B
2001 2003
2003 1999
1990 2001
1995 2010
2004 1996
I want to check if there are values similar between the two columns regardless of the rows and place it in a new column (SIMILAR)
This is the output that I would like to have
A B SIMILAR
2001 2003 2003
2003 1999 2001
1990 2001
1995 2010
2004 1996
Thank you
IIUC you can use isin:
df[df['A'].isin(df['B'])]['A'].values
If by "similar" you mean equal, I'd solve this with the isin method. I'm also assuming that the order of values in the new column does not matter.
>>> df['SIMILAR'] = df.loc[df['A'].isin(df['B']), 'A']
>>> df
A B SIMILAR
0 2001 2003 2001.0
1 2003 1999 2003.0
2 1990 2001 NaN
3 1995 2010 NaN
4 2004 1996 NaN
To find the duplicated values you can do something like this:
duplicateRowsDF = pdData[pdData.duplicated()]
print("Duplicate Rows except first occurrence based on all columns are :")
print(duplicateRowsDF)
The response should be somethin like this:
SIMILAR
2003
2001
Then you just use this new data to create a new colum
pdData["Similar"] = duplicateRowsDF
Code-golfing with set intersection (assumes a standard range index):
df['C'] = pd.Series([*{*df.A} & {*df.B}])
A B C
0 2001 2003 2001.0
1 2003 1999 2003.0
2 1990 2001 NaN
3 1995 2010 NaN
4 2004 1996 NaN
