generator comprehension

Question

May i just a quick question.what does generator comprehension do?And how does it work?i couldnt find a tutorial about it. Thanks in advance

Answer 1

A generator comprehension is the lazy version of a list comprehension.

It is just like a list comprehension except that it returns an iterator instead of the list ie an object with a next() method that will yield the next element.

If you are not familiar with list comprehensions see here and for generators see here.

Answer 2

List/generator comprehension is a construct which you can use to create a new list/generator from an existing one.

Let's say you want to generate the list of squares of each number from 1 to 10. You can do this in Python:

>>> [x**2 for x in range(1,11)]
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

here, range(1,11) generates the list [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], but the range function is not a generator before Python 3.0, and therefore the construct I've used is a list comprehension.

If I wanted to create a generator that does the same thing, I could do it like this:

>>> (x**2 for x in xrange(1,11))
<generator object at 0x7f0a79273488>

In Python 3, however, range is a generator, so the outcome depends only on the syntax you use (square brackets or round brackets).

Answer 3

Do you understand list comprehensions? If so, a generator expression is like a list comprehension, but instead of finding all the items you're interested and packing them into list, it waits, and yields each item out of the expression, one by one.

>>> my_list = [1, 3, 5, 9, 2, 6]
>>> filtered_list = [item for item in my_list if item > 3]
>>> print filtered_list
[5, 9, 6]
>>> len(filtered_list)
3
>>> # compare to generator expression
... 
>>> filtered_gen = (item for item in my_list if item > 3)
>>> print filtered_gen  # notice it's a generator object
<generator object at 0xb7d5e02c>
>>> len(filtered_gen) # So technically, it has no length
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: object of type 'generator' has no len()
>>> # We extract each item out individually. We'll do it manually first.
... 
>>> filtered_gen.next()
5
>>> filtered_gen.next()
9
>>> filtered_gen.next()
6
>>> filtered_gen.next() # Should be all out of items and give an error
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>> # Yup, the generator is spent. No values for you!
... 
>>> # Let's prove it gives the same results as our list comprehension
... 
>>> filtered_gen = (item for item in my_list if item > 3)
>>> gen_to_list = list(filtered_gen)
>>> print gen_to_list
[5, 9, 6]
>>> filtered_list == gen_to_list
True
>>>

Because a generator expression only has to yield one item at a time, it can lead to big savings in memory usage. Generator expressions make the most sense in scenarios where you need to take one item at a time, do a lot of calculations based on that item, and then move on to the next item. If you need more than one value, you can also use a generator expression and grab a few at a time. If you need all the values before your program proceeds, use a list comprehension instead.