How can I make Perl wait for child processes started in the background with system()?

Question

I have some Perl code that executes a shell script for multiple parameters, to simplify, I'll just assume that I have code that looks like this:

for $p (@a){
    system("/path/to/file.sh $p&");
}

I'd like to do some more things after that, but I can't find a way to wait for all the child processes to finish before continuing.

Converting the code to use fork() would be difficult. Isn't there an easier way?

Answer 1

Converting to fork() might be difficult, but it is the correct tool. system() is a blocking call; you're getting the non-blocking behavior by executing a shell and telling it to run your scripts in the background. That means that Perl has no idea what the PIDs of the children might be, which means your script does not know what to wait for.

You could try to communicate the PIDs up to the Perl script, but that quickly gets out of hand. Use fork().

Answer 2

You are going to have to change something, changing the code to use fork is probably simpler, but if you are dead set against using fork, you could use a wrapper shell script that touches a file when it is done and then have your Perl code check for the existence of the files.

Here is the wrapper:

#!/bin/bash

$*

touch /tmp/$2.$PPID

Your Perl code would look like:

for my $p (@a){
    system("/path/to/wrapper.sh /path/to/file.sh $p &");
}
while (@a) {
    delete $a[0] if -f "/tmp/$a[0].$$";
}

But I think the forking code is safer and clearer:

my @pids;
for my $p (@a) {
    die "could not fork" unless defined(my $pid = fork);\
    unless ($pid) { #child execs
        exec "/path/to/file.sh", $p;
        die "exec of file.sh failed";
    }
    push @pids, $pid; #parent stores children's pids
}

#wait for all children to finish
for my $pid (@pids) {
    waitpid $pid, 0;
}

Answer 3

Using fork/exec/wait isn't so bad:

my @a = (1, 2, 3);
for my $p (@a) {
   my $pid = fork();
   if ($pid == -1) {
       die;
   } elsif ($pid == 0) {
      exec '/bin/sleep', $p or die;
   }
}
while (wait() != -1) {}
print "Done\n";