find columns in Dataframe where every row has a value [duplicate]

Question

how can I find the columns which have a value in every row My pd.DataFrame looks like this:

     A    B    C    D
1    1    2    3    2
2    2    2         1
3         3    2    1

i would like to get this output

     B    D
1    2    2
2    2    1
3    3    1

how can i detect the right columns?

Thank you, R

I've marked this as a duplicate of a question asking for columns containing any null value. You just need to use the negative of this, i.e. the ~ operator. — jpp
– jpp, Commented Oct 12, 2018 at 11:37

jezrael · Accepted Answer · 2018-10-12 11:29:48Z

2

Use if no values are missing values:

df1 = df.loc[:, df.notna().all()]
#oldier pandas versions
#df1 = df.loc[:, df.notnull().all()]

print (df1)
   B  D
1  2  2
2  2  1
3  3  1

Explanation:

Compare no missing values by by notna:

print (df.notna())
       A     B      C     D
1   True  True   True  True
2   True  True  False  True
3  False  True   True  True

Check if all values in columns are True by DataFrame.all:

print (df.notna().all())
A    False
B     True
C    False
D     True
dtype: bool

If no values are empty strings compare by DataFrame.ne (!=):

df = df.loc[:, df.ne('').all()]

answered Oct 12, 2018 at 11:29

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

1 Answer 1