Why replacing values in numpy array does not always work

Question

I am trying to replace/overwrite values in a array using the following commands:

import numpy as np
test = np.array([[4,5,0],[0,0,0],[0,0,6]])
test
Out[20]:
array([[4., 5., 0.],
       [0., 0., 0.],
       [0., 0., 6.]])

test[np.where(test[...,0] != 0)][...,1:3] = np.array([[10,11]])
test
Out[22]:
array([[4., 5., 0.],
       [0., 0., 0.],
       [0., 0., 6.]])

However, as one can see in Out22, the array test has not been modified. So I am concluding that it is not possible to simply overwrite a part of a array or just few cells.

Nevertheless, in other contexts, it is possible to overwrite few cells of a array. For example, in the below code:

test = np.array([[1,2,0],[0,0,0],[0,0,3]])
test
Out[11]:
array([[1., 2., 0.],
       [0., 0., 0.],
       [0., 0., 3.]])

test[test>0]
Out[12]:
array([1., 2., 3.])

test[test>0] = np.array([4,5,6])
test
Out[14]:
array([[4., 5., 0.],
       [0., 0., 0.],
       [0., 0., 6.]])

Therefore, my 2 questions:

1- Why does the first command

test[np.where(test[...,0] != 0)][...,1:3] = np.array([10,11])

does not allow modifying the array test ? Why does not it allow accessing the array cells and overwrite them?

2- How could I make it work considering that for my code I would need to select the cells using the command above?

Many thanks!

The test[where...] step makes a copy. The following [...]=... modifies that copy, not the original. A view pass the changes through, but not a copy. — hpaulj
– hpaulj, Commented Nov 3, 2020 at 16:46

Daniel F · Accepted Answer · 2020-11-03 10:22:04Z

I'll do you one up. This does work:

test[...,1:3][np.where(test[...,0] != 0)] = np.array([[10,11]])

array([[ 4, 10, 11],
       [ 0,  0,  0],
       [ 0,  0,  6]])

Why? It's the combination of two factors - numpy indexing and .__setitem__ calls.

The python interpreter sort of reads lines backwards. And when it gets to =, it tries to call .__setitem__ on the furthest thing to the left. __setitem__ is (hopefully) a method of the object, and has two inputs, the target and the indices (whatever is between [...] just before it).

a[b] = c  #is intepreted as
a.__setitem__(b, c)

Now, when we index in numpy we have three basic ways we can do it.

slicing (returns views)
'advanced indexing' (returns copies)
'simple indexing' (also returns copies)

One major difference between "advanced" and "simple" indexing is that a numpy array's __setitem__ function can interpret advanced indexes. And views mean the data addresses are the same, so we don't need __setitem__ to get to them.

So:

test[np.where(test[...,0] != 0)][...,1:3] = np.array([[10,11]])  #is intepreted as

(test[np.where(test[...,0] != 0)]).__setitem__( slice([...,1:3]), 
                                                np.array([[10,11]]))

But, since np.where(test[...,0] != 0) is an advanced index, (test[np.where(test[...,0] != 0)]) returns a copy, which is then lost because it is never assigned. It does take the elements we want and set them to [10,11], but the result is lost in the buffer somewhere.

If we do:

test[..., 1:3][np.where(test[..., 0] != 0)] = np.array([[10, 11]]) #is intepreted as

(test[..., 1:3]).__setitem__( np.where(test[...,0] != 0), np.array([[10,11]]) )

test[...,1:3] is a view, so it still points to the same memory. Now setitem looks for the locations in test[...,1:3] that correspond to np.where(test[...,0] != 0), and set them equal to np.array([[10,11]]). And everything works.

You can also do this:

test[np.where(test[...,0] != 0), 1:3] = np.array([10, 11])

Now, since all the indexing is in one set of brackets, it's calling test.__setitem__ on those indices, which sets the data correctly as well.

Even simpler (and most pythonic) would be:

test[test[...,0] != 0, 1:3] = np.array([10,11])

Collectives™ on Stack Overflow

Why replacing values in numpy array does not always work

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related