Issue applying textblob to a dataframe series

Question

After splitting my dataset into train, test, and validation sets I have a x_validation set which is a set of strings. Calling x_validation.head() gives:

0    this drink is making my throat hurt more and need to convince corey to go to jacks mannequin concert obvs will be in need of advil
1                         there gonna be movie on no can see it not even the trailers hate thinking about it as it is ll have breakdown
2                                                 the wire on my braces is too long and is cutting through my cheek farrrrrrrk it hurts
3                                             finally have uploaded my documentary to an external site message me for link and password
4                                        lovely national day today hour children parade and hour citizens parade with ju jitsu training

It has something like 15,000 strings total. I'm trying to create a new list tbresult containing the sentiment polarity scores of each string as calculated by TextBlob:

tbresult = [TextBlob(i).sentiment.polarity for i in x_validation]

This gives me the following error:

TypeError: The `text` argument passed to `__init__(text)` must be a string, not <class 'float'>

I'm confused because when I do the following,

lst = [x for x in x_validation]
TextBlob(lst[0]).sentiment.polarity

it works, I get 0.5. I'm confused where this float type is coming from in the error. How do I do this properly?

4.Pi.n · Accepted Answer · 2020-07-03 18:49:03Z

1

Try to remove rows contains a float value, or use .isna().sum() rather than using dropna.

def remove_floats(row):
  if isinstance(row, str):
    return row
  else:
    return None

df = pd.DataFrame({'col':['balh_1', 'blah_2', 1.0, 'blah_3']})

for key in df:
  df[key] = df[key].apply(remove_floats)

df.dropna(inplace=True)

df

     col
0   balh_1
1   blah_2
3   blah_3

edited Jul 3, 2020 at 18:49

answered Jul 3, 2020 at 18:24

4.Pi.n

1,1617 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Alex Over a year ago

Appreciate it. I did this, as well as x_validation.to_frame(), then x_validation.applymap(str). I then used your function for good measure, though when I called x_validation.dropna, it didn't appear to remove anything. Either way, things work! Thanks for your help.

Collectives™ on Stack Overflow

Issue applying textblob to a dataframe series

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related