1

Is there a straight forward way of filling nan values in a numpy array when the left and right non nan values match?

For example, if I have an array that has False, False , NaN, NaN, False, I want the NaN values to also be False. If the left and right values do not match, I want it to keep the NaN

4
  • Try to use numpy.isnan() function Commented Sep 4, 2019 at 21:04
  • 1
    False is a boolean. nan us a float. Commented Sep 4, 2019 at 21:39
  • If you check …. a = np.array([np.NaN, False, np.NaN, True]) … will yield …. array([nan, 0., nan, 1.]) …. which apparently you don't have. What is your actual array? Commented Sep 4, 2019 at 23:41
  • What's the array's dtype and shape? And in the result? Commented Sep 5, 2019 at 0:16

2 Answers 2

2

Your first task is to reliably identify the np.nan elements. Because it's a unique float value, testing isn't trivail. np.isnan is the best numpy tool.

To mix boolean and float (np.nan) you have to use object dtype:

In [68]: arr = np.array([False, False, np.nan, np.nan, False],object)                                        
In [69]: arr                                                                                                 
Out[69]: array([False, False, nan, nan, False], dtype=object)

converting to float changes the False to 0 (and True to 1):

In [70]: arr.astype(float)                                                                                   
Out[70]: array([ 0.,  0., nan, nan,  0.])

np.isnan is a good test for nan, but it only works on floats:

In [71]: np.isnan(arr)                                                                                       
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-71-25d2f1dae78d> in <module>
----> 1 np.isnan(arr)

TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

In [72]: np.isnan(arr.astype(float))                                                                         
Out[72]: array([False, False,  True,  True, False])

You could test the object array (or a list) with a helper function and a list comprehension:

In [73]: def foo(x): 
    ...:     try: 
    ...:         return np.isnan(x) 
    ...:     except TypeError: 
    ...:         return x 
    ...:                                                                                                     
In [74]: [foo(x) for x in arr]                                                                               
Out[74]: [False, False, True, True, False]

Having reliably identified the nan values, you can then apply the before/after logic. I'm not sure if it's easier with lists or array (your logic isn't entirely clear).

Sign up to request clarification or add additional context in comments.

Comments

0

See also Most efficient way to forward-fill NaN values in numpy array

Pandas offers the interpolate method on data frames. You could use it like this:

interpolate(limit_direction='both').to_numpy().flatten()

A much more specialized method is given below, it allows controlling the B-Spline interpolation parameters, extrapolation and smoothing:

https://github.com/DomTomCat/signalProcessing/blob/main/interpolateNonFiniteValues.py

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.