using python 'with' statement with iterators?

Question

Hi, I'm using Python 2.5. I'm trying to use this 'with' statement.

from __future__ import with_statement
a = []
with open('exampletxt.txt','r') as f:
    while True:
        a.append(f.next().strip().split())
print a

The contents of 'exampletxt.txt' are simple:

a
b

In this case, I get the error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/python-7036sVf.py", line 5, in <module>
    a.append(f.next().strip().split())
StopIteration

And if I replace f.next() with f.read(), it seems to be caught in an infinite loop.

I wonder if I have to write a decorator class that accepts the iterator object as an argument, and define an __exit__ method for it?

I know it's more pythonic to use a for-loop for iterators, but I wanted to implement a while loop within a generator that's called by a for-loop... something like

def g(f):
    while True:
        x = f.next()
        if test1(x):
            a = x
        elif test2(x):
            b = f.next()
            yield [a,x,b]

a = []
with open(filename) as f:
    for x in g(f):
        a.append(x)

Answer 1

Raising StopIteration is what an iterator does when it gets to the end. Normally the for statement catches it silently and continues to the else clause, but if it's being iterated manually as in your case then the code has to be prepared to handle the exception itself.

Answer 2

Your while loop doesn't end, but the file does so it raises a StopIteration exception when there is nothing else to iterate to.

Answer 3

You don't have any terminating condition in any of your while loops, so you keep returning until you get StopIteration exception which you don't handle.

Answer 4

You can always rewrite while-with-explicit-next loops. When you have an explicit next, you're just looking ahead one token.

Generally, loops of this form can be rewritten.

def g(f):
    while True:
        x = f.next()
        if test1(x):
            a = x
        elif test2(x):
            b = f.next()
            yield [a,x,b]

You can always replace a look-ahead next by buffering a value.

def g(f):
    prev, a = None, None
    for x in f:
        if test2(prev)
            yield [ a, prev, x ]
        elif test1(x):
            a = x
        prev= x