Manipulating an array in python

Question

I have an numpy array that is obtained by reading an image.

data=band.ReadAsArray(0,0,rows,cols)

Now the problem is that while using loops to manipulate the data it took around 13 min. how can I reduce this time. is there any other solution.

sample code

for i in range(rows):
     for j in range(cols):
          if data[i][j]>1 and data[i][j]<30:
                 data[i][j]=255
          elif data[i][j]<1:
                  data[i][j]=0
          else:
              data[i][j]=1

it takes too long. any short method

How big are your lists? And how big is your image? I'm pretty sure is not the "for" loops causing the slow... Use a profiler or timeit() to identify your problem. I've played with numpy's "arrays" and personally I had no performance issues (13 minutes is outrageous even on old hardware). — StefanNch
– StefanNch, Commented Jul 12, 2014 at 14:32
My array has got 6920 cols and 6540 rows. its an array of an satellite image. — Davis Nhemaphuki
– Davis Nhemaphuki, Commented Jul 13, 2014 at 16:39

Ffisegydd · Accepted Answer · 2014-07-12 14:26:48Z

3

With numpy you can use a mask to select all elements with a certain condition, as shown in the code example below:

import numpy as np

a = np.random.random((5,5))
a[a<0.5] = 0.0

print(a)
# [[ 0.          0.94925686  0.8946333   0.51562938  0.99873065]
#  [ 0.          0.          0.          0.          0.        ]
#  [ 0.86719795  0.          0.8187514   0.          0.72529116]
#  [ 0.6036299   0.9463493   0.78283466  0.6516331   0.84991734]
#  [ 0.72939806  0.85408697  0.          0.59062025  0.6704499 ]]

If you wished to re-write your code then it could be something like:

data=band.ReadAsArray(0,0,rows,cols)
data[data >= 1 & data<30] = 255
data[data<1] = 0

answered Jul 12, 2014 at 14:26

Ffisegydd

54k16 gold badges151 silver badges125 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

DSM · Accepted Answer · 2014-07-12 14:26:48Z

Instead of looping, you can assign using a boolean array to select the values you're interested in changing. For example, if we have an array

>>> a = np.array([[0.1, 0.5, 1], [10, 20, 30], [40, 50, 60]])
>>> a
array([[  0.1,   0.5,   1. ],
       [ 10. ,  20. ,  30. ],
       [ 40. ,  50. ,  60. ]])

We can apply your logic with something like

>>> anew = np.empty_like(a)
>>> anew.fill(1)
>>> anew[a < 1] = 0
>>> anew[(a > 1) & (a < 30)] = 255
>>> anew
array([[   0.,    0.,    1.],
       [ 255.,  255.,    1.],
       [   1.,    1.,    1.]])

This works because of how numpy indexing works:

>>> a < 1
array([[ True,  True, False],
       [False, False, False],
       [False, False, False]], dtype=bool)
>>> anew[a < 1]
array([ 0.,  0.])

Note: we don't really need anew-- you can act on a itself -- but then you have to be careful about the order you apply things in case your conditions and the target values overlap.

Note #2: your conditions mean that if there's an element of the array which is exactly 30, or anything greater, it will become 1, and not 255. That seems a little odd, but it's what your code does, so I reproduced it.

Collectives™ on Stack Overflow

Manipulating an array in python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related