Best practice for detecting when group of threads are done

Question

What's the best practice for detecting when a whole group of threads are done processing? I have a process that will query a [long running] web service for an arbitrary number of objects, and then needs to take a transactional action when all of them have completed successfully. I am currently running them asynchronously, using delegates from the .Net thread pool. Running them synchronously, defeats the purpose of running them on multiple threads... How else can I detect when ALL are finished? I though of using a coounter, (aka COM referece count) incrementing it for reach thread that starts and decrementing it in a callback function, and of keeping a dynamic list with a reference to each thread in it, to explicitly keep track of each one as they complete, but both of these solutions seem kinda kludgy...

Thanks to all... Based on yr suggestions, and on need to pass an object instance to the aynchronous thread, (represented by ref variable uPL below), I am using the following code... NOTE: IEEDAL.GetUsagePayloadReadings(uPL1) is the remote web service call

    foreach (MeterChannel chgChan in chgChs)
        foreach (UsagePayload uPL in chgChan.IntervalPayloads)
        {
            ManualResetEvent txEvnt = new ManualResetEvent(false);
            UsagePayload uPL1 = uPL;
            ThreadPool.QueueUserWorkItem(
                delegate(object state)
                    {
                        if (!uPL1.HasData)
                            IEEDAL.GetUsagePayloadReadings(uPL1);
                        UsageCache.PersistPayload(uPL1);
                        SavePayLoadToProcessFolder(uPL1);
                        txEvnt.Set();
                    } );
            waitHndls.Add(txEvnt);
        }
    WaitHandle.WaitAll(waitHndls.ToArray());

Answer 1

There are a couple of options.

A Semaphore lets you have a counter used across all of your threads. If you know exactly how many there are, you can use this to check for when they're completed.

Another potential alternative is to have a ResetEvent for each thread. Just set the event at the "end" of the thread, and check them in your main thread. This may be more difficult, though.

Answer 2

You can use WaitHandle.WaitAll.

Answer 3

I don't know if it's a "best practice" or not, but the System.Threading.ManualResetEvent and System.Threading.WaitHandle classes are two classes that I find indispensable when threads need to signal one another. Here is a rudimentary example of how to use them with ThreadPool threads.

    List<WaitHandle> handles = new List<WaitHandle>();
    for ( int iii = 0; iii < 10; iii++ ) {
        ManualResetEvent transactionEvent = new ManualResetEvent( false );
        ThreadPool.QueueUserWorkItem( delegate( object state ) {
            // Do your work here...
            transactionEvent.Set();
        } );
        handles.Add( transactionEvent );
    }
    WaitHandle.WaitAll( handles.ToArray() );

Answer 4

Dim WaitAllEvents(1) As AutoResetEvent

Dim thread1 As Thread
Dim thread2 As Thread

thread1 = New Thread(AddressOf Thread1Worker)
thread2 = New Thread(AddressOf Thread2Worker)

WaitAllEvents(0) = New AutoResetEvent(False)
WaitAllEvents(1) = New AutoResetEvent(False)

thread1.Start()
thread2.Start()

'Main thread will wait until all instances of AutoResetEvent 
'have become signaled with a call to Set()

WaitHandle.WaitAll(WaitAllEvents)

Console.WriteLine("All threads done exiting main thread")
thread2 = Nothing
thread1 = Nothing

'...
'...

Private Sub Thread1Worker()
    Thread.Sleep(5000)
    Console.WriteLine("Thread1 done")
    WaitAllEvents(0).Set() 
End Sub

Private Sub Thread2Worker()
    Thread.Sleep(3000)
    Console.WriteLine("Thread2 done")
    WaitAllEvents(1).Set() 
End Sub

Answer 5

The best way I've seen to do this is to use a countdown latch. There's a very good description, including C# source, in the May 2007 issue of MSDN Magazine.