Pandas to_numpy() results in an array of lists. How do I get a 2D numpy array from this?

Question

I have a column of lists, and each list contains the same number of values.

If I do

df['column'].to_numpy()

I get an array of lists:

array([list([0, 4688, 11, 43486, 40508, 13, 5,...
       list([0, 40928, 17707, 22705, 9, 38312, 2..
       list([0, 6766, 368, 3551, 28837,..
      dtype=object)

How do I get a 2D array instead?

That's odd. Can you share the content of your df["column"] ? — k88
– k88, Commented May 12, 2020 at 3:42
Show how to construct this dataframe. Does any list have a non-numeric element? — Mad Physicist
– Mad Physicist, Commented May 12, 2020 at 3:56

Ehsan · Accepted Answer · 2020-05-12 03:59:08Z

7

You can do this:

np.array(df['column'].tolist())

Or you can simply stack them:

np.stack(df['column'].to_numpy())

This will stack your lists on top of each other and output is a 2-D array. You have to make sure lists are of the same length. Numpy arrays are rectangular.

edited May 12, 2020 at 3:59

answered May 12, 2020 at 3:45

Ehsan

12.5k2 gold badges24 silver badges36 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Mad Physicist Over a year ago

Technically true, but likely doesn't address the root of the problem

Ehsan Over a year ago

@MadPhysicist Not sure what you mean. It is one of many solutions. I added another one that deals with pandas first if that is what you mean by root of the problem :D

Mad Physicist Over a year ago

Getting a dataframe with lists is usually weird

Mad Physicist Over a year ago

That being said, +1 for the cleanest solution based on the information provided

Ehsan Over a year ago

@MadPhysicist I agree a better practice is to not include same length lists in a column, but I can see applications were having it sounds reasonable too. Thanks for the upvote :)

|

Dishin Goyani · Accepted Answer · 2020-05-12 03:57:46Z

1

You can use list list comprehension

>>> df
                 A
0  [0, 1, 2, 3, 4]
1  [0, 1, 2, 3, 4]
2  [0, 1, 2, 3, 4]

>>> np.array([x for x in df['A']])
array([[0, 1, 2, 3, 4],
       [0, 1, 2, 3, 4],
       [0, 1, 2, 3, 4]])

answered May 12, 2020 at 3:57

Dishin Goyani

7,7533 gold badges33 silver badges42 bronze badges

Comments

Saugat Pudasaini · Accepted Answer · 2020-05-12 04:07:41Z

0

data=np.array(df['column']).reshape(2,2)

It will convert your desire columns into array

for 2d array you can use reshape function on numpy But you should have proper value in data to corvert eg (2,2) is used if you have 4 numbers of data it indicate (row,columns)

edited May 12, 2020 at 4:07

answered May 12, 2020 at 4:02

Saugat Pudasaini

212 bronze badges

2 Comments

Ehsan Over a year ago

This actually should return 1-D array of lists again. OP wants a 2-D array.

Ehsan Over a year ago

The shape is not necessarily (2,2) either. The example in the question shows you that it is actually not of shape (2,2). You might have been confusing shape with dimensions.

Collectives™ on Stack Overflow

Pandas to_numpy() results in an array of lists. How do I get a 2D numpy array from this?

3 Answers 3

7 Comments

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

7 Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related