What is the Perl version of a Python iterator?

Question

I am learning Perl at my work and enjoying it. I usually do my work in Python but boss wants Perl.

Most of the concepts in Python and Perl match nicely: Python dictionary=Perl hash; Python tuple=Perl list; Python list=Perl array; etc.

Question: Is there a Perl version of the Python form of an Iterator / Generator?

An example: A Classic Python way to generate the Fibonacci numbers is:

#!/usr/bin/python

def fibonacci(mag):
     a, b = 0, 1
     while a<=10**mag:
         yield a
         a, b = b, a+b

for number in fibonacci(15):  
     print "%17d" % number

Iterators are also useful if you want to generate a subsection of a much larger list as needed. Perl 'lists' seem more static - more like a Python tuple. In Perl, can foreach be dynamic or is only based on a static list?

The Python form of Iterator is a form that I have gotten used to, and I do not find it documented in Perl... Other than writing this in loops or recursively or generating a huge static list, how do I (for ex) write the Fibonacci subroutine it in Perl? Is there a Perl yield that I am missing?

Specifically -- how do I write this:

#!/usr/bin/perl
use warnings; use strict; # yes -- i use those!

sub fibonacci {
   # What goes here other than returning an array or list? 
}

foreach my $number (fibonacci(15)) { print $number . "\n"; }

Thanks in advance to being kind to the newbie...

Answer 1

There's a good practical example here and a PDF article here... but I'm too rusty in Perl to try to implement your challenge directly (as you'll see, both the example and the approach in the PDF use a less direct approach).

Answer 2

There is a similar method to produce a Iterator / Generator, but it is not a "first class citizen" as it is on Python.

In Perl, if you do not see what you want (after a MANDATORY trip to CPAN FIRST!), you can roll your own that is similar to a Python iterator based on Perl closures and an anonymous subroutine.

Consider:

use strict; use warnings;

sub fibo {
    my ($an, $bn)=(1,0);
    my $mag=(shift || 1);
    my $limit=10**$mag;
    my $i=0;

    return sub {
        ($an, $bn)=($bn, $an+$bn);      
        return undef if ($an >=$limit || wantarray );
        return $an;
    }
}

my $num;
my $iter=fibo(15);
while (defined($num=$iter->()) ) { printf "%17d\n", $num; }

The sub fibo maintains a Perl closure that allows persistent variables to be maintained. You can do the same by having a module, similar to C / C++. Inside fibo an anonymous subroutine does the work of returning the next data item.

To quote from the Perl Bible "You will be miserable until you learn the difference between scalar and list context" -- p 69 (A highly recommended book btw...)

In this case, the annon sub only returns a single value. The only looping mechanism that I know of in Perl that can work in scalar context is while; The others try to fill the list before proceeding I think. Therefor, if you called the anon sub in list context, it will dutifully return the next fibonacci number, unlike Python's for iterators, and the loop would terminate. That is why I put the return undef if .... wantarray because it does not work in list context as written.

There are ways to fix that. Indeed, you can write subroutines that act like map foreach etc but it is not as straightforward as Python's yield. You will need an additional function to use inside a foreach loop. The tradeoff is the Perl approach has tremendous power and flexibility.

You can read more about Perl iterators in Mark Jason Dominus' excellent book "Higher Order Perl" Chapter 4 is all about Interators brian d foy also has an excellent article on Interators in the Perl Review.

Answer 3

The concept of an iterator is a little different in Perl. You basically want to return a one-use subroutine "closed" over the persistent variables.

use bigint;
use strict;
use warnings;

sub fibonacci {
    my $limit = 10**( shift || 0 );
    my ( $a, $b ) = ( 0, 1 );
    return sub { 
        return if $a > $limit;
        ( my $r, $a, $b ) = ( $a, $b, $a + $b );
        return $r;
    };
}
my $fit = fibonacci( 15 );
my $n = 0;
while ( defined( my $f = $fit->())) { 
     print "F($n): $f\n";
     $n++;
}

And if you don't like the while loop, then here is two shots at some syntactic sugar, which basically accomplish a each item loop.:

sub iterate ($$) {
    my $iter   = shift;
    my $action = shift;
    while ( defined( my $nextval = $iter->())) { 
        local *_ = \$nextval;
        $action->( $_ );
    }
    return;
}

iterate fibonacci( 15 ) => sub { print "$_\n"; };

sub iter (&$) { 
    my $action = shift;
    my $iter   = shift;
    while ( defined( my $nextval = $iter->())) { 
        local *_ = \$nextval;
        $action->( $_ );
    }
    return;
}

iter { print "$_\n" } fibonacci( 15 );

Answer 4

The excellent Higher-Order Perl book (available for free at the specified link) contains a lot of information on related topics, and in particular has a whole chapter on iterators. By "higher order" the author implies using Perl's abilities as a functional language with first-class functions to implement all kinds of cool stuff. It really is a very good book - I read most of it, and the chapters on iterators and streams are terrific. I highly recommend to at least skim through it if you plan to write Perl code.

Answer 5

For an even more flexible solution than Python's generators, I have written the module List::Gen on CPAN which provides random access lazy generator arrays:

use List::Gen;

my $fib; $fib = cache gen {$_ < 2  ? $_ : $$fib[$_ - 1] + $$fib[$_ - 2]};

say "@$fib[0 .. 15]";  #  0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610

Since generators pretend to be arrays, they can mix seamlessly with normal perl code. There is also an object oriented approach:

my $fib; $fib = cache gen {$_ < 2 ? $_ : $fib->get($_ - 1) + $fib->get($_ - 2)};

say join ' ' => $fib->slice(0 .. 15);

In each case, the generator is lazy, calculating nothing upon creation, and then calculating only those values required to satisfy the slices. The recursive definition of the Fibonacci sequence calls itself many times, so the cache function is used to make sure each value is only calculated once.

You can also use generators as iterators:

while (my $num = $fib->next) {
    last if $num > 10**15;
    print "$_\n";
}

$fib->next can also be written $fib->(). Since the generator is still random access, you can $fib->reset() or $fib->index = 10;

Let me know if you have any questions.

Update:

I have released a new version of the module (0.80) that makes it easier to use iterative algorithms in generators. Here is an example that closely mirrors the OP's example:

use List::Gen '*';

sub fibonacci {
    my $limit   = 10**shift;
    my ($x, $y) = (0, 1);

    While {$_ < $limit} gather {
        ($x, $y) = ($y, take($x) + $y)
    }
}

say for @{fibonacci 15};

if you use bigint; before or at the top of the sub, you can of course:

say for @{fibonacci 400}; # or more

Answer 6

In this case, memoization can be used.

use strict;
use warnings;

use Memoize;
memoize('fib');

foreach my $i (1..15) {
  print "$i -> ",fib($i),"\n";
}

sub fib {
  my $n = shift;
  return $n if $n < 2;
  fib($n-1) + fib($n-2);
}