views:

46

answers:

1

All, pls see below test code about itertools.tee:

    li = [x for x in range(10)]
    ite = iter(li)
==================================================
    it = itertools.tee(ite, 5)
    >>> type(ite)
    <type 'listiterator'>
    >>> type(it)
    <type 'tuple'>
    >>> type(it[0])
    <type 'itertools.tee'>
    >>> 

    >>> list(ite)
    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
    >>> list(it[0])          # here I got nothing after 'list(ite)', why?
    []
    >>> list(it[1])
    []
====================play again===================
    >>> ite = iter(li)
    it = itertools.tee(ite, 5)
    >>> list(it[1])
    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
    >>> list(it[2])
    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
    >>> list(it[3])
    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
    >>> list(it[4])
    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
    >>> list(ite)
    []                       # why I got nothing? and why below line still have the data?   
    >>> list(it[0])
    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
    >>> list(it[0])
    []
====================play again===================    
    >>> ite = iter(li)
    itt = itertools.tee(it[0], 5)    # tee the iter's tee[0].
    >>> list(itt[0])
    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
    >>> list(itt[1])
    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
    >>> list(it[0])
    []                               # why this has no data?
    >>> list(it[1])
    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
    >>> list(ite)
    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]  

My quesion is

1.how tee works and why sometimes the original iter 'has data' and other time it has not? 
2.can I keep an iter deep copy as a "status seed" to keep the raw iterator status and tee it to use later? 
3.can I swap 2 iters or 2 itertools.tee?

Thanks!

+2  A: 

tee takes over the original iterator; once you tee an iterator, discard the original iterator since the tee owns it (unless you really know what you're doing).

You can make a copy of a tee with the copy module:

import copy, itertools
it = [1,2,3,4]
a, b = itertools.tee(it)
c = copy.copy(a)

... or by calling a.__copy__().

Beware that tee works by keeping track of all of the iterated values that have been consumed from the original iterator, which may still be consumed by copies.

For example,

a = [1,2,3,4]
b, c = itertools.tee(a)
next(b)

At this point, the tee object underlying b and c has read one value, 1. It's storing that in memory, since it has to remember it for when c is iterated. It has to keep every value in memory until it's consumed by all copies of the tee.

The consequence of this is that you need to be careful with "saving state" by copying a tee. If you don't actually consume any values from the "saved state" tee, you're going to cause the tee to keep every value returned by the iterator in memory forever (until the copied tee is discarded and collected).

Glenn Maynard
thanks #Glenn, so tee can be think as a data buffer could conduct as iter, right? so it is may not suitable for large dataset,and is there a way can duplicate the "pure" iter for a single sequence? I know deep copy can not work on an iter.
user478514
No, there's no way in general to copy an iterator. Iterators *can* expose `__copy__` as `tee` instances do, but usually do not.
Glenn Maynard