Python: Apply function to every entry in numpy 3d array

Question

I would like to apply a (more complex?) function on my 3d numpy array with the shape x,y,z = (4,4,3). Let's assume I have the following array:

array = np.arange(48)
array = array.reshape([4,4,3])

Now I would like to call the following function on each point of the array:

p(x,y,z) = a(z) + b(z)*ps(x,y)

Let's assume a and b are the following 1d arrays, respectively ps a 2d array.

a = np.random.randint(1,10, size=3)
b = np.random.randint(1,10, size=3)
ps = np.arrange(16)
ps = ps.reshape([4,4])

My intuitive approach was to loop over my array and call the function on each point. It works, but of course it's way too slow:

def calcP(a,b,ps,x,y,z):
    p = a[z]+b[z]*ps[x,y]
    return p

def stupidLoop(array, a, b, ps, x, y, z):
    dummy = array
    for z in range (0, 3):
        for x in range (0, 4):
            for y in range (0, 4):
                dummy[x,y,z]=calcP(a,b,ps,x,y,z)
    return dummy

updatedArray=stupidLoop(array,a, b, ps, x, y, z)

Is there a faster way? I know it works with vectorized functions, but I cannot figure it out with mine.

I didn't actually try it with these numbers. It's just to exemplify my problem. It comes from the Meteorology world and is a little more complex.

Do you mean ps = ps.reshape([4,4])? There is some error with the ndarrays you've shared — yatu
– yatu, Commented Mar 5, 2019 at 10:29
IIRC you can use the map function to iterate through np.arrays as well so 'map(f, array)' might just do the trick? — meissner_
– meissner_, Commented Mar 5, 2019 at 10:32
I think you want dummy = a[:, np.newaxis, np.newaxis] + b[:, np.newaxis, np.newaxis] * ps (according to the snippet, because in the formula you wrote first the indices are in different order, so it would be a + b * p[:, :, np.newaxis]). — javidcf
– javidcf, Commented Mar 5, 2019 at 10:36
@jdehesa and @yatu thanks for pointing out the mistakes. The new variable (in my case updatedArray should have the exact same dimensions as my old variable array. And why should I add more dimensions to a, b and ps with np.newaxis? — Sev
– Sev, Commented Mar 5, 2019 at 10:46
@meissner_, The map function will only iterate over the outer dimension of a numpy array. It does not understand the inner dimensions. — fountainhead
– fountainhead, Commented Mar 5, 2019 at 11:10

Jan Zhen · Accepted Answer · 2019-03-06 01:31:04Z

2

Vectorize the loop, and use broadcasting:

a.reshape([1,1,-1]) + b.reshape([1,1,-1]) * ps.reshape([4,4,1])

EDIT:

Thanks @NilsWerner for offering a more common way in comment:

a + b * ps[:, :, None]

edited Mar 6, 2019 at 1:31

answered Mar 5, 2019 at 11:24

Jan Zhen

736 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Nils Werner Over a year ago

A more common way is to write a + b * ps[:, :, None]

Jan Zhen Over a year ago

Thanks @NilsWerner . It's better, and I will edit my answer.

fountainhead · Accepted Answer · 2019-03-05 11:16:02Z

0

You can do this using numpy.fromfunction():

import numpy as np

a = np.random.randint(1,10, size=3)
b = np.random.randint(1,10, size=3)
ps = np.arange(16)
ps = ps.reshape([4,4])

def calcP(x,y,z,a=a,b=b,ps=ps):
    p = a[z]+b[z]*ps[x,y] + 0.0
    return p

array = np.arange(48)
array = array.reshape([4,4,3])

updatedArray = np.fromfunction(calcP, (4,4,3), a=a,b=b,ps=ps, dtype=int)
print (updatedArray)

Notice that I've modified your function calcP slightly, to take kwargs. Also, I've added 0.0, to ensure that the output array will be of floats and not ints.

Also, notice that the second argument to fromfunction() merely specifies the shape of the grid, over which the function calcP() is to be invoked.

Output (will vary each time due to randint):

[[[  8.   5.   3.]
  [  9.   6.  12.]
  [ 10.   7.  21.]
  [ 11.   8.  30.]]

 [[ 12.   9.  39.]
  [ 13.  10.  48.]
  [ 14.  11.  57.]
  [ 15.  12.  66.]]

 [[ 16.  13.  75.]
  [ 17.  14.  84.]
  [ 18.  15.  93.]
  [ 19.  16. 102.]]

 [[ 20.  17. 111.]
  [ 21.  18. 120.]
  [ 22.  19. 129.]
  [ 23.  20. 138.]]]

edited Mar 5, 2019 at 11:16

answered Mar 5, 2019 at 11:03

fountainhead

3,7421 gold badge11 silver badges18 bronze badges

13 Comments

Sev Over a year ago

This is getting so close! Thanks already. But I made some minor mistakes in the dimensions. Let's say the shape should be (3,4,4). So, 3 layers with each 4*4. But when I try to change it to array.reshape([3,4,4]) and also updatedArray = np.fromfunction(calcP, (3,4,4), a=a,b=b,ps=ps, dtype=int) I get an IndexError: index 3 is out of bounds for axis 1 with size 3

Sev Over a year ago

Okay I think I got it. I needed to adapt my function as well: def calcP(z,x,y,a=a,b=b,ps=ps): p = a[z]+b[z]*ps[x,y] return p

fountainhead Over a year ago

@Sev: You're welcome. If this was helpful, pls upvote the answer (click on the grey up arrow icon). Also pls click on the tick mark icon to "Accept" the answer so others will know this has been answered..

Nils Werner Over a year ago

Note that this solution will be slower than the broadcasting, and by an order of magnitude.

fountainhead Over a year ago

@NilsWerner:How so? And what exactly is being broadcasted in that solution?

|

Collectives™ on Stack Overflow

Python: Apply function to every entry in numpy 3d array

2 Answers 2

2 Comments

13 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

13 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related