How to concatenate 2 lists of numpy arrays

Question

I have 2 lists l1, l2 that contain 5 numpy arrays each of the following shapes:

[(32, 84, 84, 1), (32,), (32,), (32,), (32, 84, 84, 1)]

The result I want is achieved using:

[np.concatenate(item) for item in zip(l1, l2)]

The resulting shapes are (which are the expected / desired result)

[(64, 84, 84, 1), (64,), (64,), (64,), (64, 84, 84, 1)]

However when I try to concatenate them directly using np.concatenate(l1, l2), I get an error:

ValueError: could not broadcast input array from shape (32,84,84,1) into shape (32)

So I expand dims for the middle 3 arrays in each l1 and l2, the resulting shapes are:

[(32, 84, 84, 1), (1, 32), (1, 32), (1, 32), (32, 84, 84, 1)]

I then try to concatenate again:

np.concatenate((l1, l2))

And the resulting shapes are:

[(32, 84, 84, 1),
 (1, 32),
 (1, 32),
 (1, 32),
 (32, 84, 84, 1),
 (32, 84, 84, 1),
 (1, 32),
 (1, 32),
 (1, 32),
 (32, 84, 84, 1)]

And trying with np.concatenate((l1, l2), axis=1) results in the following error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3418, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-111-0928bf6cff84>", line 1, in <module>
    ccc = np.concatenate((aaa, bbb), axis=1)
  File "<__array_function__ internals>", line 5, in concatenate
numpy.AxisError: axis 1 is out of bounds for array of dimension 1

So, is there a way to do it without a for loop?

can you give the sample input for l1 and l2? Is there a particular reason you want to avoid a for loop? — Chris Doyle
– Chris Doyle, Commented Jan 6, 2021 at 1:22
If your desired result is a list of arrays with different shapes, why are you playing tricks to avoid the obvious loop? — hpaulj
– hpaulj, Commented Jan 6, 2021 at 1:22
@Chris Doyle You may use np.random.random() and specify the shapes I mentioned and try yourself, it won't make a difference what the values are. — watch-this
– watch-this, Commented Jan 6, 2021 at 1:25
But what makes you think a for loop is not efficient? list comprehensions in python are pretty fast. Sure you can choose other things like map(np.concatenate, zip(l1, l2)) but under the hood its still going to have to iterate all the items returned from zip and apply the np function to them. you seem to be under the impression that for loops must just be slow. if you run it and find a bottle neck or performance problem them re-evaluate the issue. but given your current scenario your always going to have to iterate over the items and for loops are pretty efficient at that — Chris Doyle
– Chris Doyle, Commented Jan 6, 2021 at 1:34

manju-dev · Accepted Answer · 2021-01-06 11:08:04Z

1

An ndarray is a (usually fixed-size) multidimensional container of items of the same type and size

So you can't convert your list (of numpy arrays) into ndarray with np.array() call.

Also, np.concatenate returns ndarray, so because of the above reason, you can't apply that function on your list as well (but applying to each item in the list works, like you have already found out).

So if you have to avoid explicit looping for some reason, use map as pointed out by @ChrisDoyle in comments:

map(np.concatenate, zip(l1, l2))

The N-dimensional array (ndarray)

edited Jan 6, 2021 at 11:08

answered Jan 6, 2021 at 10:47

manju-dev

4462 silver badges9 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

watch-this Over a year ago

thanks, it won't matter much because as @Chris Doyle indicated in the comments, it would be using a for loop under the hood to iterate over them, so it's the same thing.

Collectives™ on Stack Overflow

How to concatenate 2 lists of numpy arrays

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related