Pandas apply to create multiple columns, using multiple columns as input

Question

I am trying to use a function to create multiple outputs, using multiple columns as inputs. Here's my attempt:

df = pd.DataFrame(np.random.randint(0,10,size=(6, 4)), columns=list('ABCD'))
df.head()

    A   B   C   D
0   8   2   5   0
1   9   9   8   6
2   4   0   1   7
3   8   4   0   3
4   5   6   9   9

def some_func(a, b, c):
    return a+b, a+b+c

df['dd'], df['ee'] = df.apply(lambda x: some_func(a = x['A'], b = x['B'], c = x['C']), axis=1, result_type='expand')

df.head()

   A    B   C   D   dd  ee
0   8   2   5   0   0   1
1   9   9   8   6   0   1
2   4   0   1   7   0   1
3   8   4   0   3   0   1
4   5   6   9   9   0   1

The outputs are all 0 for the first new column, and all 1 for the next new column. I am interested in the correct solution, but I am also curious about why my code resulted this way.

jezrael · Accepted Answer · 2020-05-25 05:46:39Z

3

You can assign to subset ['dd','ee']:

def some_func(a, b, c):
    return a+b, a+b+c

df[['dd','ee']] = df.apply(lambda x: some_func(a = x['A'], 
                                               b = x['B'], 
                                               c = x['C']), axis=1, result_type='expand')
print (df)
   A  B  C  D  dd  ee
0  4  7  7  3  11  18
1  2  1  3  4   3   6
2  4  7  6  0  11  17
3  0  9  1  1   9  10
4  5  6  5  9  11  16
5  3  2  4  9   5   9

If possible, better/ fatser is use vectorized solution:

df = df.assign(dd = df.A + df.B, ee = df.A + df.B + df.C)

answered May 25, 2020 at 5:46

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

SantoshGupta7 Over a year ago

can df assign be used for complicated functions? I was using a minimal example, but my function irl involves processing in a library which uses a gpu.

jezrael Over a year ago

@SantoshGupta7 - It depends of function, mainly if possible processing by arrays. It means if you can add input all values of columns and output is new Series (unfortunately many custom function cannot do it :()

wong.lok.yin · Accepted Answer · 2020-05-25 06:18:58Z

1

Just to explain the 0, 1 part. 0 and 1 are actually the column names of

df.apply(lambda x: some_func(a = x['A'], b = x['B'], c = x['C']), axis=1, result_type='expand')

That is

x = df.apply(lambda x: some_func(a = x['A'], b = x['B'], c = x['C']), axis=1, result_type='expand')
a, b = x
print(a)    # first column name
print(b)    # second column name

output:
0
1

Finally, you assign

df['dd'], df['ee'] = 0, 1

results in

   A    B   C   D   dd  ee
0   8   2   5   0   0   1
1   9   9   8   6   0   1
2   4   0   1   7   0   1
3   8   4   0   3   0   1
4   5   6   9   9   0   1

answered May 25, 2020 at 6:18

wong.lok.yin

8891 gold badge6 silver badges11 bronze badges

Comments

SantoshGupta7 · Accepted Answer · 2021-02-14 08:37:57Z

0

Alternative way:

df['dd'], df['ee'] = zip(*df.apply(lambda x: some_func(x['A'], x['B'], x['C]) )

answered Feb 14, 2021 at 8:37

SantoshGupta7

6,32718 gold badges73 silver badges142 bronze badges

Collectives™ on Stack Overflow

Pandas apply to create multiple columns, using multiple columns as input

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related