Are Python list comprehensions the same thing as map/grep in Perl?

Question

I was having some trouble grokking the list comprehension syntax in Python, so I started thinking about how to achieve the same thing in Perl, which I'm more familiar with. I realized that the basic examples (taken from this page) can all be done in Perl with map or grep.

E.g.

(python)                            (perl)                  
S = [x**2 for x in range(10)]       @S = map { $_**2 } ( 0..9 );
V = [2**i for i in range(13)]       @V = map { 2**$_ } ( 0..12 );
M = [x for x in S if x % 2 == 0]    @M = grep { $_ % 2 == 0 } @S;

So is "list comprehension" just a fancy term for "map and/or filter a list" or is there more to it?

Answer 1

You are correct: a list comprehension is essentially just syntactic sugar for map and filter (terms from the functional programming world).

Hopefully this sample code demonstrates their equality:

>>> # Python 2
>>> [x**2 for x in range(10)] == map(lambda x: x**2, range(10))
True
>>> [2**i for i in range(13)] == map(lambda x: 2**x, range(13))
True
>>> S = [x**2 for x in range(10)]
>>> [x for x in S if x % 2 == 0] == filter(lambda x: x % 2 == 0, S)
True

Note that this is only valid in Python 2.X, as SilentGhost pointed out in the comment. To make this compatible with Python 3, you'll have to wrap the calls to map or filter in the list constructor, because map and filter have been updated to return iterators, not lists.

>>> # Python 3
>>> [x**2 for x in range(10)] == list(map(lambda x: x**2, range(10)))
True
>>> [2**i for i in range(13)] == list(map(lambda x: 2**x, range(13)))
True
>>> S = [x**2 for x in range(10)]
>>> [x for x in S if x % 2 == 0] == list(filter(lambda x: x % 2 == 0, S))
True

Answer 2

List comprehensions are more powerful than map or filter as they allow you to abstractly play with lists.

It also more convenient to use them when your maps are further nested with more maps and filter calls.

Answer 3

Yes, they are basically the same.

In fact Python also has a map function:

S = map(lambda x: x**2, range(10))

is the same as your first examples above. However, the list comprehension syntax is strongly preferred in Python. I believe Guido has been quoted as saying he regrets introducing the functional syntax at all.

However, where it gets really interesting is in the next evolution of list comprehensions, which is generators. These allow you to return an iterator - rather than processing the whole list at once, it does a single iteration and then returns, so that you don't have to hold the whole list in memory at the same time. Very powerful.

Answer 4

Yes. The power of the Python syntax is that the same syntax (within round rather than square brackets) is also used to define generators, which produce sequences of values on demand.

Answer 5

They're the "pythonic" version for mapping and filtering sequences, but they allow to do some others things, like flattening a (fixed level) nested list, for example:

[j for i in nested_list for j in i]

Another thing that you cannot do with a regular map and a lambda expression is to structurally decompose the iterating values, for example:

[(x%y)*z for x,y,z in list_with_triplets_of_ints]

of course there are workarounds like:

aux = lambda x,y,z: (x%y)*z
map(lambda t: aux(*t), list_with_triplets_of_ints)

but when the transformation you need to apply is already defined, then usually it's just simpler to use a map, like in:

map(int, list_of_str_values)

rather than

[int(i) for i in list_of_str_values]

Answer 6

List comprehensions also flatten out things:

For example:

[(x, y) for x in xrange(10) if x%2 == 0 for y in xrange(20) if x!=y]

If you used nested maps here, you'd have to use concat (summing the lists) too.