Python pandas - new column based on other columns (String)

Question

I couldn't find it in stackoverflow, so I wanted to ask the question.

Let's assume that i have two columns: A, B in data frame, which consist of just a bunch of words, and i want to create a new column C which is just TRUE/FALSE based on the following rule:

 If word in B = word in A + 'ing', then it's True or vice versa
 If word in B = word in A + 'ment', then it's True of vice versa.

so I defined the following function:

def parts_of_speech(s1, s2):
    return s1+'ing'==s2 or s1+'ment'==s2 or s1+s1[-1]+'ing'==s2

For instance

  A              B            C
Engage         Engagement   True
Go             Going        True
Axe            Axis         False
Management     Manage       True

I tried the following:

df['C']=df.apply(lambda x: parts_of_speech(x.A, x.B) or 
                           parts_of_speech(x.B, x.A) )

or

df['C']=df.apply(parts_of_speech(df['A'], df['B']) or 
                           parts_of_speech(df['A'], df['B']) )

I get the same error:

A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

I don't know what i did incorrectly. Is there an easy fix for this?

any help would be greatly appreciated.

s1+s1[-1]+'ing'==s2 is not correct. You will get something like Manageeing. Use s1[:-1] + 'ing' instead. — Trenton McKinney
– Trenton McKinney, Commented Sep 19, 2019 at 17:49

Marat · Accepted Answer · 2019-09-19 17:41:53Z

2

.apply works with columns by default. The only change needed in your example is to add axis=1 to apply to rows:

df['C']=df.apply(lambda x: parts_of_speech(x.A, x.B) or parts_of_speech(x.B, x.A),
                 axis=1)

answered Sep 19, 2019 at 17:41

Marat

15.8k3 gold badges44 silver badges53 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Quang Hoang · Accepted Answer · 2019-09-19 17:47:38Z

1

For your sample data:

# make B the longer words
df[['A','B']] = np.sort(df[['A','B']])

# split by suffixes
df['B'].str.extract('(\w+)(ment|ing)$',expand=True)[0].eq(df['A'])

Or use your approach, but vectorized:

# make B the longer words
df[['A','B']] = np.sort(df[['A','B']])

df['A-ing'] = df['A'] + 'ing'
df['A-ment'] = df['A'] + 'ment'

df.iloc[:,-2].eq(df['A']).all(1)

Output:

0     True
1     True
2    False
3     True
dtype: bool

edited Sep 19, 2019 at 17:47

answered Sep 19, 2019 at 17:45

Quang Hoang

151k11 gold badges64 silver badges86 bronze badges

3 Comments

Marat Over a year ago

There are some more complex cases, like hop + ing = hopping. I guess the question is not about the language processing part

Quang Hoang Over a year ago

Sure, but then it's about the logic, and your answer doesn't work as well.

Marat Over a year ago

> I guess the question is not about the language processing part

Collectives™ on Stack Overflow

Python pandas - new column based on other columns (String)

2 Answers 2

Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related