views:

648

answers:

7

Given x number of arrays, each with a possibly different number of elements, how can I iterate through all combinations where I select one item from each array?

Example:

[   ]   [   ]   [   ]
 foo     cat      1
 bar     dog      2
 baz              3
                  4

Returns

[foo]   [cat]   [ 1 ]
[foo]   [cat]   [ 2 ]
  ...
[baz]   [dog]   [ 4 ]

I'm doing this in Perl, btw.

A: 

a triple for loop? (scream)

dfa
Nothing like O(N^3) to make your day! (But hey, could be worse... could be O(3^N)...)
Amber
are you joking? there are N x M x P permutations...
dfa
which is O(max(N, M, P)^3)
bdonlan
O(N^3)??? but you are generating N^3 permutations, the issue is the constraint that you can only generate permutations of three elements.
nlucaroni
sometimes you have no choices... if he needs all the permutations, you cannot optimize nothing :)
dfa
No, unless you elaborate more. Given `x` arrays, I would need x-tuple nested for-loop structure. I can't hard-code a triple for loop like that, in my case, because I don't know how many different fields I will have.
erjiang
There's no need for a nested loop. My module does it for an arbitrary number of sets without any nested loops.
brian d foy
I **really** don't understand this "hate" against nested for loops
dfa
@dfa: Nested loops don't scale. I don't understand your "love" for them.
brian d foy
they are simple, they comunicate well what the code does, they doesn't consume memory, you don't add another dependency to your project, etc is not a love, it is the most simple solution that works for this simple problem
dfa
Nested loops aren't at all simple, and they don't communicate well. It's more structure than you need, and much too inflexible.
brian d foy
+2  A: 

A simple recursive solution for an arbitrary number of lists:

sub permute {
  my ($first_list, @remain) = @_;

  unless (defined($first_list)) {
    return []; # only possibility is the null set
  }

  my @accum;
  for my $elem (@$first_list) {
    push @accum, (map { [$elem, @$_] } permute(@remain));
  }

  return @accum;
}

A not-so-simple non-recursive solution for an arbitrary number of lists:

sub make_generator {
  my @lists = reverse @_;

  my @state = map { 0 } @lists;

  return sub {
    my $i = 0;

    return undef unless defined $state[0];

    while ($i < @lists) {
      $state[$i]++;
      last if $state[$i] < scalar @{$lists[$i]};
      $state[$i] = 0;
      $i++;
    }

    if ($i >= @state) {
      ## Sabotage things so we don't produce any more values
      $state[0] = undef;
      return undef;
    }

    my @out;
    for (0..$#state) {
      push @out, $lists[$_][$state[$_]];
    }

    return [reverse @out];
  };
}

my $gen = make_generator([qw/foo bar baz/], [qw/cat dog/], [1..4]);
while ($_ = $gen->()) {
  print join(", ", @$_), "\n";
}
bdonlan
+1 very elegant, thanks for sharing
dfa
Note that there's some unnecessary allocation here - it can be optimized a bit further. But this general approach is what you want :)
bdonlan
It's not really the approach that you want. Avoid recursion in Perl, and don't create the whole thing in memory.
brian d foy
Non-recursive variant added :)
bdonlan
permute() has a bug. Every inner list contains an empty array ref at the end.
FM
Here's a good tutorial on iterators:http://www.perlmonks.org/?node_id=451278
BrianH
A: 

There's one method I thought of first that uses a couple for loops and no recursion.

  1. find total number of permutations
  2. loop from 0 to total_permutations-1
  3. observe that, by taking the loop index modulus the number of elements in an array, you can get every permutations

Example:

Given A[3], B[2], C[3],

for (index = 0..totalpermutations) {
    print A[index % 3];
    print B[(index / 3) % 2];
    print C[(index / 6) % 3];
}

where of course a for loop can be substituted to loop over [A B C ...], and a small part can be memoized. Of course, recursion is neater, but this might be useful for languages in which recursion is severely limited by stack size.

erjiang
I think it's the same as three nested loops, except with this approach you also spend time on doing the math in the process.
Andrew Y
a triple for loop is far efficient and easy to read
dfa
+13  A: 

My Set::CrossProduct module does exactly what you want. Note that you aren't really looking for permutations, which is the ordering of the elements in a set. You're looking for the cross product, which is the combinations of elements from different sets.

My module gives you an iterator, so you don't create it all in memory. You create a new tuple only when you need it.

brian d foy
+1 ... I had not seen your post when I cooked up my attempt at repeating the functionality of this module (I did not know it existed either).
Sinan Ünür
A: 

Recursive and more-fluent Perl examples (with commentary and documentation) for doing the Cartesian product can be found at http://www.perlmonks.org/?node_id=7366

Example:

sub cartesian {
    my @C = map { [ $_ ] } @{ shift @_ };

    foreach (@_) {
        my @A = @$_;

        @C = map { my $n = $_; map { [ $n, @$_ ] } @C } @A;
    }

    return @C;
}
erjiang
A: 

Here is something that seems to work for some definition of "work". It is very likely to be slower than the nested loops, but I have not done any measurement.

#!/usr/bin/perl

use strict;
use warnings;

use List::Util qw( max );

my @arrays = (
    [ qw( foo bar baz) ],
    [ qw( cat dog ) ],
    [ 1, 2, 3, 4 ],
);

my $count = 1;
$count *= @{ $_ } for @arrays;

my $max = max map { scalar @{ $_} } @arrays;
my @generators = map { generate( $_, $max ) } @arrays;

my @output;

for ( 1 .. $count ) {
    print join($", map { $_->() } @generators), "\n";
}

sub generate {
    my @array = @{ shift @_ };
    my $max   = shift;
    my $index = 0;
    if ( $max == @array ) {
        return sub { $array[ $index++ % @array ] };
    }
    else {
        my $this = -1;
        return sub {
            unless ( ++$this < $max ) {
                $this = 0;
                $index += 1;
            }
            return $array[ $index % @array ];
        };
    }
}

Output:

C:\Temp> fgh
foo cat 1
foo cat 2
foo cat 3
foo cat 4
bar dog 1
bar dog 2
bar dog 3
bar dog 4
baz cat 1
baz cat 2
baz cat 3
baz cat 4
foo dog 1
foo dog 2
foo dog 3
foo dog 4
bar cat 1
bar cat 2
bar cat 3
bar cat 4
baz dog 1
baz dog 2
baz dog 3
baz dog 4
Sinan Ünür