How to delete rows in python pandas DataFrame using regular expressions?

Question

I have a pattern:

patternDel = "( \\((MoM|QoQ)\\))";

And I want to delete all rows in pandas dataframe where column df['Event Name'] matches this pattern. Which is the best way to do it? There are more than 100k rows in dataframe.

Just to be clear, when you import columns with that string, they will be converted to "( \((MoM|QoQ)\))" unless they are raw strings. It would be helpful to include a sample of the data in order to get the best answer. — shawnheide
– shawnheide, Commented Oct 9, 2016 at 23:00

Bob Haffner · Accepted Answer · 2016-10-09 22:44:42Z

66

str.contains() returns a Series of booleans that we can use to index our frame

patternDel = "( \\((MoM|QoQ)\\))"
filter = df['Event Name'].str.contains(patternDel)

I tend to keep the things we want as opposed to delete rows. Since filter represents things we want to delete we use ~ to get all the rows that don't match and keep them

df = df[~filter]

answered Oct 9, 2016 at 22:44

Bob Haffner

8,5231 gold badge40 silver badges44 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Joshua Stafford Over a year ago

Thanks, this works great. Probably shouldn't use the keyword "filter" as a variable name though, just to avoid confusion.

Collectives™ on Stack Overflow

How to delete rows in python pandas DataFrame using regular expressions?

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related