np.where() not returning expected indices in numpy python

Question

I have this wind data set that consists of wind speed in m/s and I would like to to count periods of nonzero data in the time series.

Each period of nonzero data would count as one "weather event".
I would also like to know where those events are located within the data set (i.e., the indices).

One way to do this is to count the first 0 before each group of non-zero data in the series to determine the number of events and then adding each index value by one to get the location of the events.

# create mock data. 
d=np.zeros([209])
d1=np.random.randn(189)
d2=np.zeros(9)
d3=np.random.randn(281)
d4=np.zeros(27)
d5=np.random.randn(21)
d6=np.zeros(155)
d7=np.random.randn(58)
mock_data=np.concatenate((d,d1,d2,d3,d4,d5,d6,d7),axis=0)


indices=np.squeeze(np.array(np.where(mock_data!=0)))  # Returns the position vector of every non-zero in the record. 

# create a vector to store the positions of  the previous zero before each SAW event.  
dummy=np.zeros(0)
for i in range(len(indices)):
    dummy=np.append(dummy,indices[i]-1)
dummy=dummy.astype(int)
dummy=np.int64(dummy)

zerovalue=np.squeeze(np.array(np.where(mock_data[dummy]==0))) # Should return the position of the previous zero before each SAW event.  

# Add 1 to each value in zerovalue 
new_index=np.zeros(0)
for i in range(len(zerovalue)):
    new_index=np.append(date_index,zerovalue[i]+1)

However, I run into the problem of np.where() not returning the indices I'm expecting. Instead of it returning the indices that indicate where the first value of the group of non-zero data is, it's returning seemingly random indices.

For example, the first index should be 209, but I'm getting 0. Any help is greatly appreciated.

Are you always getting 0 for the first index? What do you get for other values? — Matt C
– Matt C, Commented Apr 29, 2016 at 1:30

MSeifert · Accepted Answer · 2016-04-29 02:56:57Z

Let's start by cleaning your code up:

You don't need squeeze and the array cast; just extract the first element from the where result:
```
indices = np.where(mock_data)[0]
# array([209, 210, 211, 212, 213, ... 945, 946, 947, 948])
```
NumPy can do vectorized calculations so you don't need the loop to create dummy:
```
dummy = indices - 1
```
For zero_value you can also omit squeeze and array cast; but this time you want the zero-elements so the comparison has to stay:
```
zerovalue = np.where(mock_data[dummy] == 0)[0]
# array([  0, 189, 470, 491])
```
And again NumPy vectorizes your calculations:
```
new_index = zerovalue + 1
```

So now to the interpretation, maybe you find out where it went wrong:

indices are the points where you measured some wind.
dummy is the day before you measured wind again (last day without wind)
zerovalue are the cumulative days with measured wind (you check at which indices you start measuring wind after no measureable wind). Since you stop with wind you end up neglecting the final days with wind.

If you want to find the first day with wind after at least one day without wind you need to keep your array-structure:

mock_data != 0 # boolean array where you measured wind
np.diff(mock_data != 0) # boolean array which has True between no-wind and wind.
np.where(np.diff(mock_data != 0))[0] # array with indexes where it changed
# array([208, 397, 406, 687, 714, 735, 890], dtype=int64)

That's not the final result because you have also the changes from windy to not-windy days, so you discard every second element

np.where(np.diff(mock_data != 0))[0][0::2]
# array([208, 406, 714, 890], dtype=int64)

So all your calculation can be done in one line:

np.where(np.diff(mock_data != 0))[0][0::2] + 1 # with the +1
# array([209, 407, 715, 891], dtype=int64)

and if your interested where the windy days ended just slice it with [1::2]:

np.where(np.diff(mock_data != 0))[0][1::2] + 1
# array([398, 688, 736], dtype=int64)

Collectives™ on Stack Overflow

np.where() not returning expected indices in numpy python

1 Answer 1

So all your calculation can be done in one line:

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

So all your calculation can be done in one line:

Comments

Your Answer

Sign up or log in

Post as a guest

Related