39

Example: 

from __future__ import division
import numpy as np

n = 8
"""masking lists"""
lst = range(n)
print lst

# the mask (filter)
msk = [(el>3) and (el<=6) for el in lst]
print msk

# use of the mask
print [lst[i] for i in xrange(len(lst)) if msk[i]]

"""masking arrays"""
ary = np.arange(n)
print ary

# the mask (filter)
msk = (ary>3)&(ary<=6)
print msk

# use of the mask
print ary[msk]                          # very elegant

and the results are: 

>>> 
[0, 1, 2, 3, 4, 5, 6, 7]
[False, False, False, False, True, True, True, False]
[4, 5, 6]
[0 1 2 3 4 5 6 7]
[False False False False  True  True  True False]
[4 5 6]

As you see the operation of masking on array is more elegant compared to list. If you try to use the array masking scheme on list you'll get an error: 

>>> lst[msk]
Traceback (most recent call last):
  File "<interactive input>", line 1, in <module>
TypeError: only integer arrays with one element can be converted to an index

The question is to find an elegant masking for lists.

Updates:
The answer by jamylak was accepted for introducing compress however the points mentioned by Joel Cornett made the solution complete to a desired form of my interest.

>>> mlist = MaskableList
>>> mlist(lst)[msk]
>>> [4, 5, 6]
Improve this question

6 Answers 6

Reset to default
68

If you are using numpy:

>>> import numpy as np
>>> a = np.arange(8)
>>> mask = np.array([False, False, False, False, True, True, True, False], dtype=np.bool)
>>> a[mask]
array([4, 5, 6])

If you are not using numpy you are looking for itertools.compress

>>> from itertools import compress
>>> a = range(8)
>>> mask = [False, False, False, False, True, True, True, False]
>>> list(compress(a, mask))
[4, 5, 6]
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

by far the best solution here
@jamylak : from Python 3+ it seems to be zip(), instead of izip()
@Pierre I have updated the answer now. I removed that code snippet because it may have been misleading anyway and it can be viewed through the link. Also I think since the original question used numpy it's important to highlight numpy
16

If you are using Numpy, you can do it easily using Numpy array without installing any other library:

>> a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>> msk = [ True, False, False,  True,  True,  True,  True, False, False, False]
>> a = np.array(a) # convert list to numpy array
>> result = a[msk] # mask a
>> result.tolist()
[0, 3, 4, 5, 6]
Improve this answer

Comments

8

Since jamylak already answered the question with a practical answer, here is my example of a list with builtin masking support (totally unnecessary, btw):

from itertools import compress
class MaskableList(list):
    def __getitem__(self, index):
        try: return super(MaskableList, self).__getitem__(index)
        except TypeError: return MaskableList(compress(self, index))

Usage:

>>> myList = MaskableList(range(10))
>>> myList
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> mask = [0, 1, 1, 0]
>>> myList[mask]
[1, 2]

Note that compress stops when either the data or the mask runs out. If you wish to keep the portion of the list that extends past the length of the mask, you could try something like:

from itertools import izip_longest

[i[0] for i in izip_longest(myList, mask[:len(myList)], fillvalue=True) if i[1]]
Improve this answer

10 Comments

+1 Thank you for addressing the same use of masking on lists as arrays by proposing MaskableList. It looks very interesting and works very well as my desire. A quick note is that these are a bit slower compared to array masking. I added your points as updates.
I tried your solution of a MaskableList, but I have some issues re-instantiate it. For each element in a loop I want to mask this by a new list: for i in arange(0,n): fts = MaskableList(F) sorter = argsort(A) result[i] = zip(fts[sorter],A[sorter]) but each iteration, fts[sorter] contains the same values, whereas sorter is different each time. I normally use python rather as a script language and thus I am not that familiar with objects.
@Developer: I haven't tested it specifically, but one reason that MaskableList could be considerably slower is because of the slightly expensive exception handling that's going on. Try switching the try...except around, so that it attempts to mask by default.
@MillaWell: I'm not familiar with argsort. Also, what is A, and what are the contents of F?
@JoelCornett : argsort sorts an array and returns the list of indices of the original. A=[3.5,2.0,1.1,4.0]; argsort(A) would return [2,1,0,3]. F is just a un-maskable list, let us say ["A","B","C","D"], thus zip(fts[sorter],A[sorter]) should output: {"A":1.1,"B":2.0,"C":3.5,"D":4.0}
|
4

i don't consider it elegant. It's compact, but tends to be confusing, as the construct is very different than most languages.

As Rossum has said about language design, we spend more time reading it than writing it. The more obscure the construction of a line of code, the more confusing it becomes to others, who may lack familiarity with Python, even though they have full competency in any number of other languages.

Readability trumps short form notations everyday in the real world of servicing code. Just like fixing your car. Big drawings with lots of information make troubleshooting a lot easier.

For me, I would much rather troubleshoot someone's code that uses the long form

print [lst[i] for i in xrange(len(lst)) if msk[i]]

than the numpy short notation mask. I don't need to have any special knowledge of a specific Python package to interpret it.

Improve this answer

Comments

1

You could also just use list and zip

  1. define a funcion
def masklist(mylist,mymask):
    return [a for a,b in zip(mylist,mymask) if b]
  1. use it!
n = 8
lst = range(n)
msk = [(el>3) and (el<=6) for el in lst]
lst_msk = masklist(lst,msk)
print(lst_msk)
Improve this answer

Comments

1

The following works perfectly well in Python 3:

np.array(lst)[msk]

If you need a list back as the result:

np.array(lst)[msk].tolist()
Improve this answer

1 Comment

came here to specifically get around how inefficient this approach is for large lists lol

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.