Loop and remove specific rows in python with conditional

Question

I want to remove all rows of the dataframe that don't have values 'X1' and '1' here is an example of the input and output with the following dataframe :

This is the dataframe I made:

d = {1: [0,0,0, 1,0], 2: [0,0,0, 1,0], 3:[0,0,0,1,0], 4:[0,0,1,1,0], 5:[0,0,1,'X1',0],
     6:[1,0,1,'X1',0],7:[1,0,1,'X1',0],8:[1,0,1,'X1',0],9:[1,0,'X1',0,pd.NaT]}
d = pd.DataFrame(data=d,index=range(1,6))
d

    1   2   3   4   5   6   7   8   9
1   0   0   0   0   0   1   1   1   1
2   0   0   0   0   0   0   0   0   0
3   0   0   0   1   1   1   1   1   X1
4   1   1   1   1   X1  X1  X1  X1  0
5   0   0   0   0   0   0   0   0   NaT

This is the code I made:

for column in d.columns:
    index_names = d[(d[column] == 1) | (d[column]== 'X1')]
      
index_names

    1   2   3   4   5   6   7   8   9
1   0   0   0   0   0   1   1   1   1
3   0   0   0   1   1   1   1   1   X1

It doesn't work very well! It considers just the last column and doesn't remove the fourth row. Does anyone know where i'm going wrong?

Anurag Dabas · Accepted Answer · 2021-07-28 06:51:54Z

1

you can try via boolean masking:

#your condition:
m=(d.eq(1).any(1)) | (d.eq('X1').any(1))

#Finally:
d=d[m]
#OR
d=d.loc[m]

output of d:

    1   2   3   4   5   6   7   8   9
1   0   0   0   0   0   1   1   1   1
3   0   0   0   1   1   1   1   1   X1
4   1   1   1   1   X1  X1  X1  X1  0

edited Jul 28, 2021 at 6:51

answered Jul 28, 2021 at 6:46

Anurag Dabas

24.3k9 gold badges25 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Loop and remove specific rows in python with conditional

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related