Filtering a dataframe with two conditions pandas python

Question

Quite a simple question, have tried several things with no luck.

I'm trying to isolate age ranges of customers whose birth dates fall in a certain interval.

youth = cd.loc[cd.yearofbirth.isin([1996, 1997 1998, 1999]), "SALES"]

works fine however, some of the other intervals are larger (i.e. include 30+ years) and I don't want to write them all out but rather do a less than or equal to and a more than or equal to. i.e.

youth = cd.loc[cd.yearofbirth >= 1996 | cd.yearofbirth <= 1999, "SALES"]

but this line of code gives me a error

TypeError: cannot compare a dtyped [float64] array with a scalar of type [bool]

A single inequality works fine, but its when combining it with a second which makes it error.

help greatly appreciated!

FLab · Accepted Answer · 2017-06-30 16:14:14Z

5

You need to use brackets to separate the boolean conditions. Moreover, it looks like you need the intersection (&) rather than the union (|) of conditions:

youth = cd.loc[(cd.yearofbirth >= 1996) & (cd.birth_year <= 1999), "SALES"]

This is because in Python | (or &) has precedence over >=, which explains your error.

This is also reported in the documentation:

Another common operation is the use of boolean vectors to filter the data. The operators are: | for or, & for and, and ~ for not. These must be grouped by using parentheses.

Finally, for this specific case you might not need multiple conditions at all, as you can use the between method:

youth = cd.loc[cd.yearofbirth.between(1996, 1999), "SALES"]

edited Jun 30, 2017 at 16:14

answered Jun 30, 2017 at 15:47

FLab

7,5465 gold badges40 silver badges70 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

pow Over a year ago

Thanks for this. One issue however is although no longer erroring, it doesn't quite work as i'd like. It returns ALL entries which are greater than 1996 and also all the entries that are less than 1999 as opposed to only those that lay BETWEEN the interval. any advice to amend this?

FLab Over a year ago

I guess you want to use AND condition (&) instead of OR (|), since you want the intersection

FLab Over a year ago

Glad it helped, I have edited the answer. Don't forget to upvote/accept if this solved your problem

Collectives™ on Stack Overflow

Filtering a dataframe with two conditions pandas python

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related