Pandas ValueError: Invalid fill value with a <class 'numpy.ndarray'>

Question

I have a Pandas DataFrame with a series containing dicts.

import pandas as pd

df = pd.DataFrame({
                   'c': [np.nan, 'N', np.nan, 'N'],
                   'd': [{'c':'C'}, np.nan, {'c':'C'}, np.nan]
                 })

I am attempting to fillna with:

df.fillna(np.where(df['d'].str['c'] == "C", "Y", "N"), inplace=True)

Traceback:

/Applications/Anaconda/anaconda3/lib/python3.9/site-packages/pandas/core/generic.py in fillna(self, value, method, axis, inplace, limit, downcast)
   6501             elif isinstance(value, ABCDataFrame) and self.ndim == 2:
   6502 
-> 6503                 new_data = self.where(self.notna(), value)._mgr
   6504             else:
   6505                 raise ValueError(f"invalid fill value with a {type(value)}")

ValueError: invalid fill value with a <class 'numpy.ndarray'>

BENY · Accepted Answer · 2022-07-12 21:42:14Z

2

From the doc

This value cannot be a list.

out = df.fillna({'c':(df['d'].str['c'] == "C").map({True:  "Y", False: "N"})})
Out[68]: 
   c           d
0  Y  {'c': 'C'}
1  N         NaN
2  Y  {'c': 'C'}
3  N         NaN

answered Jul 12, 2022 at 21:42

BENY

324k22 gold badges176 silver badges250 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

constantstranger · Accepted Answer · 2022-07-12 22:03:04Z

1

Here's a way to do what I believe you're trying for without using fillna():

df.c[df.c.isna()] = np.where(df.d.str['c'] == 'C', 'Y', 'N')

Output:

   c           d
0  Y  {'c': 'C'}
1  N         NaN
2  Y  {'c': 'C'}
3  N         NaN

edited Jul 12, 2022 at 22:03

answered Jul 12, 2022 at 21:56

constantstranger

9,4092 gold badges9 silver badges20 bronze badges

Collectives™ on Stack Overflow

Pandas ValueError: Invalid fill value with a <class 'numpy.ndarray'>

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related