Thread synchronization/scheduling in perl

Question

I have a perl object with a few functions in it. Each functions is called once from the main program. I would like to run some of the functions in parallel to save time. I can't run all of them together since some functions depend on the results of previous functions.

I thought of something like this:

1. For each function keep a flag that is initialized to false and is set to true by the function when it ends (e.g. the last line in func1 would be $is_func1_done = 1).

2. Start each function with a loop that waits until all the flags of the functions it depends on are true. For example: if func1 depends on func2 and func3 then:

    sub func1 {
     while (!($is_func2_done && $is_func3_done)) {
      # do nothing
     } 
     # do work
    }
   

Then I can start immediately a thread for each function, but the actual work of each function will start only when it's ready. Does this make sense? Do I need any locks here on the flags? Is using such while loops common? -- the term busy waiting comes to mind... maybe most of my CPU time will be spent on these whiles? Is there a more standard solution to this?

Answer 1

You should use $thr->join to wait for a thread to finish.

For example:

#!/usr/bin/perl

use strict; use warnings;
use threads;

my @threads = map threads->create($_), qw( func1 func2 );
$_->join for @threads;

my $thr3 = threads->create('func3');
$thr3->join;

sub func1 {
    for (1 .. 5) {
        print "func1\n";
        sleep 1 + rand 3;
    }
    return;
}

sub func2 {
    for (1 .. 5) {
        print "func2\n";
        sleep 1 + rand 2;
    }
    return;
}

sub func3 {
    print "Time to do some work\n";
}

I don't know if it is common to use such while loops: I would not.

Answer 2

Does this make sense?

Yes - each task knows its preconditions, and waits for them to be met before executing. It's one of a number of valid designs, though you might find it difficult to scale as the number of tasks grow and their interdependencies grow more complex.

Do I need any locks here on the flags?

Yes. The flags need to be shared, so that one thread can manipulate them and another see it, and shared variables need to be lock()ed to be used safely.

Is using such while loops common? -- the term busy waiting comes to mind

Sadly yes, but Don't Do That, Please. Shared variables in perl can serve as condition variables through which threads can send notifications to one another:

sub func1 {
    {
        lock(%shared_state);
        until ($shared_state{ "func2 done" } and $shared_state{ "func3 done" }) {
            cond_wait(%shared_state);
        }
    }
    # do work -- note that %shared_state is unlocked

    # now tell others that we're done
    lock(%shared_state);
    $shared_state{ "func1 done" } = 1;
    cond_broadcast(%shared_state);
    # %shared_state will be unlocked, and broadcast delivered when we leave this scope
}

When you cond_wait, the shared variable is unlocked and your thread is put to sleep. No need to busy loop.

Is there a more standard solution to this?

$thr->join, as Sinan suggests, is an easy and natural way to wait for a specific thread to finish running. Thread::Semaphore can serve a similar but more complex function (and, helpfully, can be initialized to values less than zero). A common need to "wait for these 5 threads to finish something" can be achieved with a Thread::Barrier. TMTOWTDI.