1

My program creates a numpy array within a for loop. For example it creates array with shape (100*30*10), then (160*30*10) and then may be (120*30*10) . I have to append the above to an empty numpy array such that , at the end of the loop, it will be a numpy array with shape (380*30*10) (i.e sum of 100+160+120) . The second and third dimension doesnt change in the numpy array.

How can I do the above in python. I tried the following.

np_model = np.append(np_model,np_temp1)
print("Appended model shape is",np_model.shape)
np_label = np.append(np_label,np_temp2)
print("Appended label shape is",np_label.shape)

The np_model is an empty array which I have defined as np_model = np.empty(1,30,10) and np_label as np_label = np.empty(1 ,str)

np_temp1 corresponds to array within each for loop like 100*30*10,120*30*10 etc and np_temp2 is a string with "item1","item2" etc

The np_label is a string numpy array with 1 label corresponding to np_temp1.shape[0]. But the result I get in np_model is flattened array with size 380*30*10 = 1140000

Any help is appreciated.

Improve this question
4
  • you're looking for np.stack(), specifically np.vstack() Commented Feb 17, 2020 at 12:22
  • So , how should I define the empty array? Commented Feb 17, 2020 at 12:23
  • why do you need an empty array? Commented Feb 17, 2020 at 12:30
  • You shouldn't be using array append repeatedly. Make a list first. Commented Feb 17, 2020 at 13:56

3 Answers 3

Reset to default
3

you can use numpy concatenate function, append the output numpy(s) to a list and then feed it to the concatenate function:

empty_list = []
x = np.zeros([10, 20, 4])
y = np.zeros([12, 20, 4])
empty_list.append(x)
empty_list.append(y)
z = np.concatenate(empty_list, axis=0)
print(x.shape, y.shape, z.shape)

(10, 20, 4) (12, 20, 4) (22, 20, 4)

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

But initially,my array is empty. How can I concatenate an empty numpy array with non empty array
@RamShankerG , append them to an empty list. Check my code (edited).
0

As @Nullman suggested in comment(np.vstack)

You can create empty array like this >>> np_model = np.empty((0,30,10)) 

>>> np_model = np.empty((0,30,10))

>>> a = np.random.rand(100,30,10)
>>> b = np.random.rand(160,30,10)
>>> c = np.random.rand(120,30,10)

# It can done by one-line like`np_model = np.vstack((a,b,c))`
# but i guess you have loop dependency here
>>> np_model = np.vstack((np_model,a))
>>> np_model = np.vstack((np_model,b))
>>> np_model = np.vstack((np_model,c))

>>> np_model.shape
(380, 30, 10)
Improve this answer

1 Comment

do one vstack at the end
0

To specifically answer your question towards starting with an empty array, that'd be my solution, solely using np.concatenate:

import numpy as np

# Some arrays to append in a loop
arrays = (
    np.random.rand(100, 30, 10),
    np.random.rand(160, 30, 10),
    np.random.rand(120, 30, 10)
)

# Initial empty array
array = np.zeros((0, 30, 10))

# Appending arrays in loop
for a in arrays:
    array = np.concatenate((array, a), axis=0)

# Output shape
print(array.shape)

Output:

(380, 30, 10)

Hope that helps!

----------------------------------------
System information
----------------------------------------
Platform:    Windows-10-10.0.16299-SP0
Python:      3.8.1
NumPy:       1.18.1
----------------------------------------
Improve this answer

3 Comments

list append is faster
@hpaulj I had concerns regarding "collecting" all single arrays and using just a single np.concatenate, because you'd need twice the memory of the final array (since you store all intermediate arrays in the list), which wouldn't be the case in my suggested solution - assuming, the arrays to be appended are generated (and deleted/overwritten/...) on-the-fly in the mentioned loop, which is, how I understood the question. So, depending on the amount and size of the actual arrays, "collecting" them might lead to memory issues.
With your iteration doesn't save on memory, and does a whole lot more copying.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.