Pandas dataFrame append AND add new column

Question

I am trying to append data frames together when I "find" them in a for loop, as well as add additional columns.

By this I mean:

I have an existing data frame X

a    b   c
1    2   3
5    6   7
9    10  11

I have a for loop which I "find" certain rows which meet conditional arguments. I then want to add these rows to an empty frame, with additional columns.so new data frame Y would be:

a   b   c  next
5   6   7   8

where next is the new column added. So far I have tried to add in the following way:

allInfo = pd.DataFrame(columns=[list(X), "next"])
allInfo = allInfo.append([a[a.apply(lambda x:someConditional,axis=1)], valueNext],ignore_index=True)

But this does not work. Is there a simple way to do this?

Thanks!

If you provide your actual DataFrame and the operation you're performing, I'm guessing there's a pretty simple chained operation. Tough to give a great answer with this lack of detail though. — miradulo
– miradulo, Commented Apr 11, 2017 at 14:45
I understand, but im working with very large data frames in both length and width. The code provided is the code im using. — JP1
– JP1, Commented Apr 11, 2017 at 14:52

jezrael · Accepted Answer · 2017-04-11 14:45:25Z

3

I think the best is create list of DataFrames in loop and last use concat them together with original df.

final = pd.concat([df, df1, df2, df3,...])

But better is avoid all loops in pandas, because slow.

df1 = df[df.a == 5].copy()
df1['next'] = 8
print (df1)
   a  b  c  next
1  5  6  7     8

df2 = df[df.a == 1].copy()
df2['next'] = 10
print (df2)
   a  b  c  next
0  1  2  3    10

dfs = [df, df1, df2]
final = pd.concat(dfs, ignore_index=True)
print (final)

   a   b   c  next
0  1   2   3   NaN
1  5   6   7   NaN
2  9  10  11   NaN
3  5   6   7   8.0
4  1   2   3  10.0

answered Apr 11, 2017 at 14:45

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

JP1 Over a year ago

Thanks! this is great. Im working in a loop so will have to modify slightly but is a great help.

Collectives™ on Stack Overflow

Pandas dataFrame append AND add new column

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related