How to determine if the value is ONE-BUT-LAST in a Python generator?

Question

Since generator returns values lazily, how do I determine if the value returned from a generator is one-but-last? I spent like an hour on this and can't figure it out.

Any help appreciated. Is it even possible??

Thanks, Boda Cydo!

Answer 1

You can wrap the generator in a generator that generates a sequence of pairs whose first element is a boolean telling you whether the element is the last-but-one:

def ending(generator):
    z2 = generator.next()
    z1 = generator.next()
    for x in generator:
        yield (False, z2)
        z2, z1 = z1, x
    yield (True, z2)
    yield (False, z1)

Let's test it on a simple iterator:

>>> g = iter('abcd')
>>> g
<iterator object at 0x9925b0>

You should get:

>>> for is_last_but_one, char in ending(g):
...     if is_last_but_one:
...         print "The last but one is", char
... 
The last but one is c

Too see what's happening under the hood:

>>> g = iter('abcd')
>>> for x in ending(g):
...     print x
... 
(False, 'a')
(False, 'b')
(True, 'c')
(False, 'd')

Answer 2

If you want to see arbitrary future values of an iterator without consuming them, you can wrap the iterator in a 'peekable' iterator, that can buffer future values.

import collections

class PeekIter(object):

    def __init__(self, iterable):
        self._iter = iter(iterable)
        self._peekbuf = collections.deque()

    def next(self):
        if self._peekbuf:
            return self._peekbuf.popleft()
        else:
            return self._iter.next()

    def peek(self, future=0, default=None):
        try:
            while len(self._peekbuf) <= future:
                self._peekbuf.append(self._iter.next())
            return self._peekbuf[future]
        except StopIteration:
            return default

Then, you can peek at future values without consuming them.

>>> p = PeekIter(range(3))
>>> p.peek()
0
>>> p.next()
0
>>> p.peek(0)
1
>>> p.peek(0)
1
>>> p.peek(1)
2
>>> p.peek(2)
>>> sentinel = object()
>>> sentinel
<object object at 0x28470>
>>> p.peek(1, sentinel)
2
>>> p.peek(2, sentinel)
<object object at 0x28470>