filter numpy array with row-specific criteria

Question

Suppose I have a 2d numpy array and I want to filter for the elements that pass a certain criterion on a per-row basis. For example, I want only the elements that are above the 90th percentile for their specific row. I've come up with this solution:

import numpy as np
a = np.random.random((6,5))
thresholds = np.percentile(a, 90, axis=1)
threshold_2d = np.vstack([thresholds]*a.shape[1]).T
mask = a > threshold_2d
final = np.where(mask, a, np.nan)

It works and it's vectorized but it feels a little awkward, especially the part where I create threshold_2d. Is there a more elegant way? Can I somehow automatically broadcast a condition with np.where without having to create a matching 2d mask?

gboffi · Accepted Answer · 2015-02-10 21:02:11Z

Broadcast

In [36]: np.random.seed(1023)

In [37]: a = np.random.random((6,5))

In [38]: thresholds = np.percentile(a, 90, axis=1)

In [39]: threshold_2d = np.vstack([thresholds]*a.shape[1]).T

In [40]: a>threshold_2d
Out[40]: 
array([[ True, False, False, False, False],
       [False, False,  True, False, False],
       [False,  True, False, False, False],
       [False, False, False,  True, False],
       [False, False, False, False,  True],
       [False,  True, False, False, False]], dtype=bool)

In [41]: a>thresholds[:,np.newaxis]
Out[41]: 
array([[ True, False, False, False, False],
       [False, False,  True, False, False],
       [False,  True, False, False, False],
       [False, False, False,  True, False],
       [False, False, False, False,  True],
       [False,  True, False, False, False]], dtype=bool)

In [42]:

numpy.newaxis creates an axis of length one, the resulting array view has dimensions (6,1) and can be broadcast with the a arrray.

Collectives™ on Stack Overflow

filter numpy array with row-specific criteria

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related