Why does grep on my array slice cause a stack overflow in Perl?

Question

The other day, needed to iterate over a subset of an array at a time. Initially, I did this with splice - tearing apart the array was no problem in this case. It would return N elements at a time, or whatever was left at the end of the list. All went well.

Then it turned out I needed the array later. Instead of splice, I switched to array slices. BOOM! The program exploded, sending a stack overflow everywhere. What? Why? How? I played around with it, and found a couple variants that would work. Here's the test script to demonstrate this problem:

use strict;
use warnings;

my @array = qw(a b c d e f g h i j k l m n o p q r s t u v z x c v b a s d f g a s d f a se g);
my $numPerTest = 5;

my $index = 0;
print "Separating out the subset before grepping it, good.\n";
while ($index < @array)
{
   print "Iteration $index\n";
   my @subset =  @array[$index..($index+$numPerTest)];
   @subset = grep { defined $_ } @subset;
   $index += $numPerTest;
}

$index = 0;
print "Making a copy of the array before grepping works.\n";
while ($index < @array)
{
   print "Iteration $index\n";
   my @subset = grep { defined $_ } @{[ @array[$index..($index+$numPerTest)] ]};
   $index += $numPerTest;
}

$index = 0;
print "Grepping the array slice directly, explodey!\n";
while ($index < @array)
{
   print "Iteration $index\n";
   my @subset = grep { defined $_ } @array[$index..($index+$numPerTest)];
   $index += $numPerTest;
}

(Actually, I just figured this one out, but I figured I might as well post it anyway. See if anyone else sees it. :) )

(Also, if you don't see it, this has another way of explaining why it happens.)

Answer 1

By using the slice as an lvalue, you are enlarging the array every time it is too short. Hence, it will never be out of elements.

In the first two examples, you only used it as an rvalue, thus no extra elements are created. In the third one it is an lvalue, and thus the elements are created, so that $_ can be assigned to.

This is not a behavior specific to slices: in normal array access exactly the same behavior is shown.

Answer 2

I think this chunk of code

@array[$index..($index+$numPerTest)]

is creating empty elements in your array. Then when you test

$index < @array

your @array has just gotten bigger, and your index will never be bigger than your @array size.