2

How to replace nan in numpy array into blank or empty string. I googled it and it also related nan inside a pandas dataframe instead of numpy array.

5
  • Sounds like it could be a XY problem? Why do you prefer empty strings to nans? Commented Apr 26, 2022 at 1:17
  • 4
    This is unrealistic because numpy array require all data types to be the same, and empty strings and float are not allowed to be mixed together unless you change dtype to object. Commented Apr 26, 2022 at 1:17
  • Thanks I can change data type to string. I was using this array for plot annotation so I will convert it to string. how to do that thought? Thanks Commented Apr 26, 2022 at 1:26
  • Where's the minimal reproducible example? Commented Apr 26, 2022 at 4:06
  • Does this answer your question? How to replace NaN values by Zeroes in a column of a Pandas Dataframe? Commented Apr 28, 2022 at 3:07

6 Answers 6

3

An array with np.nan will be float dtype (let's not talk about object dtypes here :))

In [274]: arr = np.array([1,2,np.nan, 4,np.nan])
In [275]: arr
Out[275]: array([ 1.,  2., nan,  4., nan])
In [277]: arr[[2,4]]
Out[277]: array([nan, nan])

We can't replace any value in such array with a string!

In [278]: arr[[2,4]] = ' '
Traceback (most recent call last):
  Input In [278] in <cell line: 1>
    arr[[2,4]] = ' '
ValueError: could not convert string to float: ''

But if we first convert the float dtype to string:

In [279]: sarr = arr.astype(str)
In [280]: sarr
Out[280]: array(['1.0', '2.0', 'nan', '4.0', 'nan'], dtype='<U32')
In [281]: sarr[[2,4]] = ' '
In [282]: sarr
Out[282]: array(['1.0', '2.0', ' ', '4.0', ' '], dtype='<U32')

In a string dtype array, 'nan' isn't special, not like it is in a float.

We have to use isnan to identify float nan:

In [283]: np.isnan(arr)
Out[283]: array([False, False,  True, False,  True])
In [284]: np.nonzero(np.isnan(arr))
Out[284]: (array([2, 4]),)

but use ordinary == to test for string 'nan':

In [285]: sarr = arr.astype(str)
In [286]: sarr == 'nan'
Out[286]: array([False, False,  True, False,  True])

Several answers suggest pandas - as in:

In [287]: S = pd.Series(arr)
In [288]: S
Out[288]: 
0    1.0
1    2.0
2    NaN
3    4.0
4    NaN
dtype: float64
In [289]: S.replace?
In [290]: S.replace(np.nan, ' ')
Out[290]: 
0    1.0
1    2.0
2       
3    4.0
4       
dtype: object

Note though the change dtype - from float to object. In this case, the series contains floats and strings.

In [292]: _.to_numpy()
Out[292]: array([1.0, 2.0, ' ', 4.0, ' '], dtype=object)
Sign up to request clarification or add additional context in comments.

1 Comment

We don't talk about dtype=object no no no
2

While the question only asks how to replace nan's with blank strings, I suspect the reason for wanting to do this is only regarding the appearance of the printed array. Not because the nan's in the floating point array are actually an issue.

In that case, you can leave the nan's in the floating point array and use numpy.set_printoptions(nanstr='') to simply have a blank space printed at any nan location, when you print the array.

See full documentation here (including many other options when printing numpy arrays): https://numpy.org/doc/stable/reference/generated/numpy.set_printoptions.html

Comments

1

You can use np.where() method to do that in this way:

a = np.array([[nan, 2], [3, nan]])
a = np.where(np.isnan(a), '', a)
print(a)

Output:

[['' '2.0']
 ['3.0' '']]

Process finished with exit code 0

Also if you want to replace it with a number value you could use np.nan_to_num() method:

a = np.array([[nan, 2], [3, nan]])
a = np.nan_to_num(a, nan=0)
print(a)

Output:

[[0. 2.]
 [3. 0.]]

Process finished with exit code 0

2 Comments

Thank you, after using np.where(), the array becomes all string? if I want to convert it back to float, how to do it? Thanks
@roudan array elements should be the same type as the array type so you should first replace blank with number value then convert it back to float. or directly convert nan to number as I mention in np.nan_to_num() method.
0

Try this:

dfCopy = df.replace(np.nan, '', regex=True)

Check out the documentation for replace here

2 Comments

I tried this method and it says numpy array doesn't have attribute of replace. I think it is used for pandas dataframe.
df is a commonly used variable name for a pandas dataframe
0

You can use built-in functions to replace particular values, for example:

import numpy as np

arr = np.array((np.nan, 1, 0, np.nan, -42))

arr[np.isnan(arr)] = -100

print(arr)

The output would be:

array([-100.,    1.,    0., -100.,  -42.])

Note: you should be careful about what value you replace np.nan with, as it should be the same type as the array (i.e. if your array is of type str you can replace with an empty string).

Comments

-1

Using fillna np methods :

Ex:

df2 = df.fillna("")

You can also convert berween numpy array to dataFram as following:

df = pd.DataFrame(numpy_array)

For more please check following: https://sparkbyexamples.com/pandas/pandas-replace-nan-with-blank-empty-string/#:~:text=Convert%20Nan%20to%20Empty%20String,in%20the%20Pandas%20DataFrame%20column.

2 Comments

sorry it is not pandas dataframe. it is numpy array so no function of fillna()
You can use the following to convert to np array:. arr = df.to_numpy()

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.