Counting None values using lambda function

Question

I have an array consisting of a bunch values, where some of them are Nan and the others are None. I want to count each of them. I can achieve this with a simple for loop as shown,

xx = np.array([2,3,4,None,np.nan,None])

count_None = 0
count_nan = 0
for i in xx:
    if i is None:
        count_None =+ 1
    if i is np.nan:
        count_nan =+ 1

I want to find out if I can achieve the same result in one line, perhaps using a lambda function. I tried writing it as so. But of course, the syntax is incorrect. Any ideas?

lambda xx: count_None =+1 if xx is None

If you can fix the title on this post, it will help other users find it. "Lambda" is misspelled as "labmda" — phrodod
– phrodod, Commented May 18, 2020 at 18:05
yea sure ofcourse :) I am open to any suggestions on better titles? — imantha
– imantha, Commented May 19, 2020 at 8:46

Joshua Varghese · Accepted Answer · 2020-05-18 04:37:57Z

3

One way of achieving it as a one liner is :

len([i for i in xx if i is None])
# or the count function
xx.count(None)

or you can use the numpy.count_nonzero:

np.count_nonzero(xx == None)

Using a lambda function, you can create a list.count() - like function:

>>> counter = lambda x,y:len([i for i in x if i == y])
>>> counter(xx,None)
2

edited May 18, 2020 at 4:37

answered May 18, 2020 at 4:32

Joshua Varghese

5,2121 gold badge18 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

hpaulj · Accepted Answer · 2020-05-18 06:44:26Z

2

This isn't a lambda but it creates a new list of just the None values and counts the length of that list.

import numpy as np
xx = np.array([2,3,4,None,np.nan,None])
print(len([elem for elem in xx if elem is None]))

if you don't need it to be in numpy you can use the list count method

xx = [2,3,4,None,np.nan,None]
print(xx.count(None))

edited May 18, 2020 at 6:44

hpaulj

233k14 gold badges260 silver badges392 bronze badges

answered May 18, 2020 at 4:36

W-B

1,2978 silver badges21 bronze badges

Comments

hpaulj · Accepted Answer · 2020-05-18 07:15:52Z

lambda is just a restricted format for creating a function. It is 'one-line' and returns a value. It should not be used for side effects. You use of counter += 1 is a side effect, so can't be use in a lambda.

A lambda that identifies the None values, can be used with map:

In [27]: alist = [2,3,4,None,np.nan,None]                                                              
In [28]: list(map(lambda x: x is None, alist))                                                         
Out[28]: [False, False, False, True, False, True]

map returns an iterable, which has to be expanded with list, or with sum:

In [29]: sum(map(lambda x: x is None, alist))                                                          
Out[29]: 2

But as others have shown, the list count method is simpler.

In [43]: alist.count(None)                                                                             
Out[43]: 2
In [44]: alist.count(np.nan)                                                                           
Out[44]: 1

An array containing None will be object dtype. Iteration on such an array is slower than iteration on the list:

In [45]: arr = np.array(alist)                                                                         
In [46]: arr                                                                                           
Out[46]: array([2, 3, 4, None, nan, None], dtype=object)

The array doesn't have the count method. Also testing for np.nan is trickier.

In [47]: arr == None                                                                                   
Out[47]: array([False, False, False,  True, False,  True])
In [48]: arr == np.nan                                                                                 
Out[48]: array([False, False, False, False, False, False])

There is a np.isnan function, but that only works for float dtype arrays.

In [51]: arr.astype(float)                                                                             
Out[51]: array([ 2.,  3.,  4., nan, nan, nan])
In [52]: np.isnan(arr.astype(float))                                                                   
Out[52]: array([False, False, False,  True,  True,  True])

phrodod · Accepted Answer · 2020-05-18 05:49:33Z

1

A third approach:

>>> nan_count, none_count = np.sum([i is np.nan for i in xx]), np.sum([i is None for i in xx])
>>> print(nan_count, none_count)
1, 2

I'd tend to prefer two lines (one for each computation), but this works. It works by adding 1 for each True value, and 0 for each False value.

Another approach if you really want to use a lambda is to use functools.reduce which will perform the sum iteratively. Here, we start with a value of 0, and add 1 for each element that evaluates true:

>>> functools.reduce(lambda x,y: x+(y is np.nan), xx, 0)
1
>>> functools.reduce(lambda x,y: x+(y is None), xx, 0)
2

edited May 18, 2020 at 5:49

answered May 18, 2020 at 5:27

phrodod

1861 silver badge9 bronze badges

Comments

tanmayjain69 · Accepted Answer · 2020-05-18 06:30:09Z

1

l= len(list(filter(lambda x:x is None, xx)))

It will return the number of NaN values. But the filter function will work with the list only. You can use this approach if you want to use lambda. I prefer using numpy function (np.count_nonzero)

answered May 18, 2020 at 6:30

tanmayjain69

1781 silver badge10 bronze badges

Collectives™ on Stack Overflow

Counting None values using lambda function

5 Answers 5

Comments

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related