How to make a repeating generator in Python

Question

How do you make a repeating generator, like xrange, in Python? For instance, if I do:

>>> m = xrange(5)
>>> print list(m)
>>> print list(m)

I get the same result both times — the numbers 0..4. However, if I try the same with yield:

>>> def myxrange(n):
...   i = 0
...   while i < n:
...     yield i
...     i += 1
>>> m = myxrange(5)
>>> print list(m)
>>> print list(m)

The second time I try to iterate over m, I get nothing back — an empty list.

Is there a simple way to create a repeating generator like xrange with yield, or generator comprehensions? I found a workaround on a Python tracker issue, which uses a decorator to transform a generator into an iterator. This restarts every time you start using it, even if you didn't use all the values last time through, just like xrange. I also came up with my own decorator, based on the same idea, which actually returns a generator, but one which can restart after throwing a StopIteration exception:

@decorator.decorator
def eternal(genfunc, *args, **kwargs):
  class _iterable:
    iter = None
    def __iter__(self): return self
    def next(self, *nargs, **nkwargs):
      self.iter = self.iter or genfunc(*args, **kwargs):
      try:
        return self.iter.next(*nargs, **nkwargs)
      except StopIteration:
        self.iter = None
        raise
  return _iterable()

Is there a better way to solve the problem, using only yield and/or generator comprehensions? Or something built into Python? So I don't need to roll my own classes and decorators?

Answer 1

I think the answer to that is "No". I'm possibly wrong. It may be that with some of the funky new things you can do with generators in 2.6 involving arguments and exception handling that would allow something like what you want. But those features are mostly intended for implementing semi-continuations.

Why do you want to not have your own classes or decorators? And why did you want to create a decorator that returned a generator instead of a class instance?

Answer 2

What about itertools.cycle.

Answer 3

Not directly. Part of the flexibility that allows generators to be used for implementing co-routines, resource management, etc, is that they are always one-shot. One run, a generator cannot be re-run. You'd have to create a new generator object.

However, you can create your own class which overrides __iter__(). It will act like a reusable generator:

def multigen(gen_func):
    class _multigen(object):
        def __init__(self, *args, **kwargs):
            self.__args = args
            self.__kwargs = kwargs
        def __iter__(self):
            return gen_func(*self.__args, **self.__kwargs)
    return _multigen

@multigen
def myxrange(n):
   i = 0
   while i < n:
     yield i
     i += 1
m = myxrange(5)
print list(m)
print list(m)

Answer 4

If you write a lot of these, John Millikin's answer is the cleanest it gets.

But if you don't mind adding 3 lines and some indentation, you can do it without a custom decorator. This composes 2 tricks:

1. [Generally useful:] You can easily make a class iterable without implementing .next() - just use a generator for __iter__(self)!

2. Instead of bothering with a constructor, you can define a one-off class inside a function.

=>

def myxrange(n):
    class Iterable(object):
        def __iter__(self):
            i = 0
            while i < n:
                yield i
                i += 1
    return Iterable()

Small print: I didn't test performance, spawning classes like this might be wasteful. But awesome ;-)