Removing rows and columns with only NaN from numpy array

Question

I am looking for a solution to this problem.

While experimenting with masks I got this error and have no clue why. It works for rows but not for columns?

import numpy as np

a = np.array(
    [[1, np.nan, 0],
    [0, np.nan, 0],
    [0, np.nan, 0],
    [np.nan, np.nan, np.nan],
    [2, np.nan, 4]])

mask_row = np.all(np.isnan(a), axis=1)
mask_column = np.all(np.isnan(a), axis=0)
print(a[~mask_row])
print(a[~mask_column])

This is the error I get for the last print statement:

IndexError: boolean index did not match indexed array along dimension 0; dimension is 5 but corresponding boolean dimension is 3

FBruzzesi · Accepted Answer · 2020-03-07 11:28:41Z

1

This is because mask_column is array([False, True, False]).

In particular mask_column.shape is (3,) i.e. 1 dimensional of size 3, while a.shape is (5,3), thus mask_column cannot be broadcasted (check numpy broadcasting for detailed broadcast explaination).

Therefore to filter nan columns you need to pass such mask as second dimension after selecting all the rows, namely:

print(a[:, ~mask_column])
[[ 1.  0.]
 [ 0.  0.]
 [ 0.  0.]
 [nan nan]
 [ 2.  4.]]

answered Mar 7, 2020 at 11:28

FBruzzesi

6,6143 gold badges19 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Removing rows and columns with only NaN from numpy array

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related