How to convert a list of numpy arrays with empty arrays into a single array?

Question

So my data looks like:

[array([0], dtype=int64),
 array([1], dtype=int64),
 array([1], dtype=int64),
 array([2], dtype=int64),
 array([3], dtype=int64),
 array([3], dtype=int64),
 array([4], dtype=int64),
 array([], dtype=int64),
 array([], dtype=int64),
 array([], dtype=int64),
 array([], dtype=int64),
 array([], dtype=int64),
 array([], dtype=int64),
 array([], dtype=int64),
 array([], dtype=int64),
 array([], dtype=int64),
 array([], dtype=int64),
 array([], dtype=int64),
 array([], dtype=int64),
 array([], dtype=int64),
 array([6], dtype=int64) ...

While using np.concatenate(list_1) does concatenate the arrays but skips the empty arrays. And in tthe resultant array 6 is the next element to 4 and intermediate empty arrays does not appear in the list. This is what np.concateneate does but I do not want it that way.

I want to combine those arrays into a single array having same length as the list but NaN values in place of empty arrays. How can I achieve that?

kabanus · Accepted Answer · 2020-05-07 18:58:43Z

2

One way:

np.concatenate([a if a.size else np.array([np.nan]) for a in array_list])

I would hazard a guess though there is probably a better way to load your data.

As a side note, concatenate does not skip any array - it concatenates the arrays, that is, puts the elements for each one after the other. Placing 0 elements is just not noticeable.

edited May 7, 2020 at 18:58

answered May 7, 2020 at 18:27

kabanus

26.3k7 gold badges48 silver badges79 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Mad Physicist Over a year ago

This is likely a very good way. You have to flag your missing items somehow one way or another

Hamza Over a year ago

There been some errors. I was able to run your snippet with following changes: np.concatenate([a if a.size else np.asarray([np.nan]) for a in array_list])

kabanus Over a year ago

@User5 np.array is enough though it shouldn't matter. I indeed had spelling mistakes (forgot size is not a function), thanks.

Hamza Over a year ago

Thanks! @kabanus that worked though and I cannot think of a better way to do it!

Mad Physicist · Accepted Answer · 2020-05-07 18:50:21Z

0

You can pre-allocate the output and assign to it directly:

result = np.full(len(list_1), np.nan)
for i, v in enumerate(list_1):
    if v.size:
        result[i] = v.item()

Alternatively, you can do

result = np.empty(len(list_1))
for i, v in enumerate(list_1):
    result[i] = v.item() if v.size else np.nan

answered May 7, 2020 at 18:50

Mad Physicist

116k29 gold badges202 silver badges292 bronze badges

Collectives™ on Stack Overflow

How to convert a list of numpy arrays with empty arrays into a single array?

2 Answers 2

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related