append data frame without a for loop in python

Question

im working with a large set of data and need a more efficient way of doing the following:

rate = [0.03,0.02,0.01]

d = {'portfolio':['abc','de','xyz'], 'A':[0,1,2],'B':[3,4,5]}

df = pd.DataFrame(data=d)

+---+-----------+---+---+
|   | portfolio | A | B |
+---+-----------+---+---+
| 0 | abc       | 0 | 3 |
| 1 | de        | 1 | 4 |
| 2 | xyz       | 2 | 5 |
+---+-----------+---+---+

essentially I have several rates that need to run through each rate scenario. and I need to add the rates to last 2 columns

currently this is the code I have:

import pandas as pd

rate = [0.03,0.02,0.01]

scenario_rate = pd.DataFrame()

for i in rate:
    d = {'portfolio':['abc','def','xyz'], 'A':[0,1,2],'B':[3,4,5]}
    df = pd.DataFrame(data=d)
    
    y = df
    y[y.columns[-2:]] += i
    y['rate'] = i
    scenario_rate = scenario_rate.append(y, ignore_index = True)



+---+-----------+------+------+------+
|   | portfolio |  A   |  B   | rate |
+---+-----------+------+------+------+
| 0 | abc       | 0.03 | 3.03 | 0.03 |
| 1 | def       | 1.03 | 4.03 | 0.03 |
| 2 | xyz       | 2.03 | 5.03 | 0.03 |
| 3 | abc       | 0.02 | 3.02 | 0.02 |
| 4 | def       | 1.02 | 4.02 | 0.02 |
| 5 | xyz       | 2.02 | 5.02 | 0.02 |
| 6 | abc       | 0.01 | 3.01 | 0.01 |
| 7 | def       | 1.01 | 4.01 | 0.01 |
| 8 | xyz       | 2.01 | 5.01 | 0.01 |
+---+-----------+------+------+------+

how do I do this more efficiently?.. perhaps without a for loop? thank you

don't even do this, scenario_rate = scenario_rate.append(y, ignore_index = True) don't do this. Instead, use a list, then use pd.concatenate at the end. Although, there is likely a better approach overall — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Sep 10, 2020 at 15:44

BENY · Accepted Answer · 2020-09-10 15:39:21Z

1

Assign it then explode

df['rate']=[rate]*len(df)
df=df.explode('rate')
df[['A','B']] = df[['A','B']].add(df['rate'],axis=0)

df
Out[62]: 
  portfolio     A     B  rate
0       abc  0.03  3.03  0.03
0       abc  0.02  3.02  0.02
0       abc  0.01  3.01  0.01
1        de  1.03  4.03  0.03
1        de  1.02  4.02  0.02
1        de  1.01  4.01  0.01
2       xyz  2.03  5.03  0.03
2       xyz  2.02  5.02  0.02
2       xyz  2.01  5.01  0.01

answered Sep 10, 2020 at 15:39

BENY

324k22 gold badges176 silver badges250 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

tez Over a year ago

thank you! this works and is quick. had to add the df= df.reset_index( drop = True)

IoaTzimas · Accepted Answer · 2020-09-10 15:56:22Z

0

The following should work:

def add_rate(somedf, somerate):
    somedf['rate']=0
    new_df=pd.concat([df['portfolio'], df[['A', 'B', 'rate']]+somerate], axis=1)
    return new_df

result=pd.concat([add_rate(df, i) for i in rate], ignore_index=True)

edited Sep 10, 2020 at 15:56

answered Sep 10, 2020 at 15:48

IoaTzimas

10.7k2 gold badges15 silver badges32 bronze badges

Collectives™ on Stack Overflow

append data frame without a for loop in python

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related