I have a dataframe with empty values in rows
How can I remove these empty values? I have already tried data.replace('', np.nan, inplace=True) and data.dropna() but that didn't change anything.
What other ways are there to drop empty rows from a dataframe?
As you have spaces in a numeric variable, I'm assuming it got read in as a string. The way I would solve this in a robust way is following the following:
data = {'lattitude': ['', '38.895118', '', '', '', '45.5234515', '', '40.764462'],
'longitude': ['', '-77.0363658', '', '', '', '-122.6762071', '', '-11.904565']}
df = pd.DataFrame(data)
Change the fields to a numeric field. errors='coerce' will change the values it can not convert to a numeric to pd.NaN.
df = df.apply(lambda x: pd.to_numeric(x, errors='coerce'))
The only thing you'll have to do now is drop the NA's
df.dropna(inplace=True)
Another possible solution is to use regular expressions. In this case it's a negative match to any character. So if the field does not have a character, it'll be caught here. Of course there are multiple regex possible here.
mask = (df['lattitude'].str.contains(r'(^\S)') & df['longitude'].str.contains(r'(^\S)'))
df = df[mask]
suppose latitude is between -90 and 90.
data = data[data['latitude'] <= 90]
this should work, no matter they are Nan or ''
Try this. Hacky, but works.
data.fillna("").replace('', pd.NA, inplace=True)
