How to replace nan in numpy array into blank or empty string. I googled it and it also related nan inside a pandas dataframe instead of numpy array.
-
Sounds like it could be a XY problem? Why do you prefer empty strings to nans?Dennis– Dennis2022-04-26 01:17:16 +00:00Commented Apr 26, 2022 at 1:17
-
4This is unrealistic because numpy array require all data types to be the same, and empty strings and float are not allowed to be mixed together unless you change dtype to object.Mechanic Pig– Mechanic Pig2022-04-26 01:17:20 +00:00Commented Apr 26, 2022 at 1:17
-
Thanks I can change data type to string. I was using this array for plot annotation so I will convert it to string. how to do that thought? Thanksroudan– roudan2022-04-26 01:26:58 +00:00Commented Apr 26, 2022 at 1:26
-
Where's the minimal reproducible example?hpaulj– hpaulj2022-04-26 04:06:48 +00:00Commented Apr 26, 2022 at 4:06
-
Does this answer your question? How to replace NaN values by Zeroes in a column of a Pandas Dataframe?Software Tester at ExpandCart– Software Tester at ExpandCart2022-04-28 03:07:03 +00:00Commented Apr 28, 2022 at 3:07
6 Answers
An array with np.nan will be float dtype (let's not talk about object dtypes here :))
In [274]: arr = np.array([1,2,np.nan, 4,np.nan])
In [275]: arr
Out[275]: array([ 1., 2., nan, 4., nan])
In [277]: arr[[2,4]]
Out[277]: array([nan, nan])
We can't replace any value in such array with a string!
In [278]: arr[[2,4]] = ' '
Traceback (most recent call last):
Input In [278] in <cell line: 1>
arr[[2,4]] = ' '
ValueError: could not convert string to float: ''
But if we first convert the float dtype to string:
In [279]: sarr = arr.astype(str)
In [280]: sarr
Out[280]: array(['1.0', '2.0', 'nan', '4.0', 'nan'], dtype='<U32')
In [281]: sarr[[2,4]] = ' '
In [282]: sarr
Out[282]: array(['1.0', '2.0', ' ', '4.0', ' '], dtype='<U32')
In a string dtype array, 'nan' isn't special, not like it is in a float.
We have to use isnan to identify float nan:
In [283]: np.isnan(arr)
Out[283]: array([False, False, True, False, True])
In [284]: np.nonzero(np.isnan(arr))
Out[284]: (array([2, 4]),)
but use ordinary == to test for string 'nan':
In [285]: sarr = arr.astype(str)
In [286]: sarr == 'nan'
Out[286]: array([False, False, True, False, True])
Several answers suggest pandas - as in:
In [287]: S = pd.Series(arr)
In [288]: S
Out[288]:
0 1.0
1 2.0
2 NaN
3 4.0
4 NaN
dtype: float64
In [289]: S.replace?
In [290]: S.replace(np.nan, ' ')
Out[290]:
0 1.0
1 2.0
2
3 4.0
4
dtype: object
Note though the change dtype - from float to object. In this case, the series contains floats and strings.
In [292]: _.to_numpy()
Out[292]: array([1.0, 2.0, ' ', 4.0, ' '], dtype=object)
1 Comment
dtype=object no no noWhile the question only asks how to replace nan's with blank strings, I suspect the reason for wanting to do this is only regarding the appearance of the printed array. Not because the nan's in the floating point array are actually an issue.
In that case, you can leave the nan's in the floating point array and use numpy.set_printoptions(nanstr='') to simply have a blank space printed at any nan location, when you print the array.
See full documentation here (including many other options when printing numpy arrays): https://numpy.org/doc/stable/reference/generated/numpy.set_printoptions.html
Comments
You can use np.where() method to do that in this way:
a = np.array([[nan, 2], [3, nan]])
a = np.where(np.isnan(a), '', a)
print(a)
Output:
[['' '2.0']
['3.0' '']]
Process finished with exit code 0
Also if you want to replace it with a number value you could use np.nan_to_num() method:
a = np.array([[nan, 2], [3, nan]])
a = np.nan_to_num(a, nan=0)
print(a)
Output:
[[0. 2.]
[3. 0.]]
Process finished with exit code 0
2 Comments
nan to number as I mention in np.nan_to_num() method.You can use built-in functions to replace particular values, for example:
import numpy as np
arr = np.array((np.nan, 1, 0, np.nan, -42))
arr[np.isnan(arr)] = -100
print(arr)
The output would be:
array([-100., 1., 0., -100., -42.])
Note: you should be careful about what value you replace np.nan with, as it should be the same type as the array (i.e. if your array is of type str you can replace with an empty string).
Comments
Using fillna np methods :
Ex:
df2 = df.fillna("")
You can also convert berween numpy array to dataFram as following:
df = pd.DataFrame(numpy_array)
For more please check following: https://sparkbyexamples.com/pandas/pandas-replace-nan-with-blank-empty-string/#:~:text=Convert%20Nan%20to%20Empty%20String,in%20the%20Pandas%20DataFrame%20column.