How to create Dataframe by adding row by row numpy arrays

Question

So I am getting bunch of arrays in streaming fashion that I want to append to a dataframe. I am trying to do it as follows

df = pd.DataFrame(columns=['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', 'y'])

outputs = get_outputs()

for xp, yp in zip(outputs, labels):
     ex = np.append(xp, yp)
     print(ex)
     print(ex.shape)
     #trying here to create row-dataframe
     exdf = pd.DataFrame(ex, columns=['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', 'y']) 
     #want to append the row to the main dataframe
     df.append(exdf)

I get output and error such

[  4.49039745  -9.63315201   7.70181465 -15.19582367  12.6580925
  -1.17788887  -5.21339655   2.6664052    2.96283174   1.22973883
   6.        ]
(11,)
...
ValueError: Shape of passed values is (11, 1), indices imply (11, 11)

How am I supposed to do it?

EDIT: after incorporating changing in answer

exdf = pd.DataFrame([ex], columns=['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', 'y'])
print(exdf)
df.append(exdf)
print(df[:1])

I receive empty dataframe

Out
          1         2         3          4  ...         8         9        10    y
0  4.490397 -9.633152  7.701815 -15.195824  ...  2.666405  2.962832  1.229739  6.0

[1 rows x 11 columns]
Empty DataFrame
Columns: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, y]
Index: []

BENY · Accepted Answer · 2019-06-01 01:11:12Z

1

Adding [] in your for loop

 for xp, yp in zip(outputs, labels):
 ex = np.append(xp, yp)
 print(ex)
 print(ex.shape)
 #trying here to create row-dataframe
 exdf = pd.DataFrame([ex], columns=['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', 'y']) 
 #want to append the row to the main dataframe
 df=df.append(exdf)

edited Jun 1, 2019 at 1:11

answered Jun 1, 2019 at 1:02

BENY

324k22 gold badges176 silver badges250 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

YohanRoth Over a year ago

I updated my post. Now for some reason dataframe is empty

BENY Over a year ago

@YohanRoth try pd.DataFrame(ex[:,None], columns=['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', 'y'])

BENY Over a year ago

@YohanRoth also , you need assign it back df=df.append(..), append in pandas is different from append for list in python , we need to assign it back

Collectives™ on Stack Overflow

How to create Dataframe by adding row by row numpy arrays

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related