What's a useful pattern for waiting for all threads to finish?

Question

I have a scenario where I will have to kick off a ton of threads (possibly up to a 100), then wait for them to finish, then perform a task (on yet another thread).

What is an accepted pattern for doing this type of work? Is it simply .Join? Or is there a higher level of abstraction nowadays?

Using .NET 2.0 with VS2008.

Answer 1

Did you look at ThreadPool? Looks like here -ThreadPool tutorial, avtor solves same task as you ask.

Answer 2

In .NET 3.5sp1 or .NET 4, the TPL would make this much easier. However, I'll tailor this to .NET 2 features only.

There are a couple of options. Using Thread.Join is perfectly acceptable, especially if the threads are all ones you are creating manually. This is very easy, reliable, and simple to implement. It would probably be my choice.

However, the other option would be to create a counter for the total amount of work, and to use a reset event when the counter reaches zero. For example:

class MyClass {
    int workToComplete; // Total number of elements
    ManualResetEvent mre; // For waiting

    void StartThreads()
    {
        this.workToComplete = 100;
        mre = new ManualResetEvent(false);

        int total = workToComplete;
        for(int i=0;i<total;++i)
        {
             Thread thread = new Thread( new ThreadStart(this.ThreadFunction) );
             thread.Start(); // Kick off the thread
        }

        mre.WaitOne(); // Will block until all work is done
    }

    void ThreadFunction()
    {
        // Do your work

        if (Interlocked.Decrement(ref this.workToComplete) == 0)
            this.mre.Set(); // Allow the main thread to continue here...
    }
}

Answer 3

What's worked well for me is to store each thread's ManagedThreadId in a dictionary as I launch it, and then have each thread pass its id back through a callback method when it completes. The callback method deletes the id from the dictionary and checks the dictionary's Count property; when it's zero you're done. Be sure to lock around the dictionary both for adding to and deleting from it.

Answer 4

I am not sure that any kind of standard thread locking or synchronization mechanisms will really work with so many threads. However, this might be a scenario where some basic messaging might be an ideal solution to the problem.

Rather than using Thread.Join, which will block (and could be very difficult to manage with so many threads), you might try setting up one more thread that aggregates completion messages from your worker threads. When the aggregator has received all expected messages, it completes. You could then use a single WaitHandle between the aggregator and your main application thread to signal that all of your worker threads are done.

public class WorkerAggregator
{
    public WorkerAggregator(WaitHandle completionEvent)
    {
        m_completionEvent = completionEvent;
        m_workers = new Dictionary<int, Thread>();
    }

    private readonly WaitHandle m_completionEvent;
    private readonly Dictionary<int, Thread> m_workers;

    public void StartWorker(Action worker)
    {
        var thread = new Thread(d =>
            {
                worker();
                notifyComplete(thread.ManagedThreadID);
            }
        );

        lock (m_workers)
        {
            m_workers.Add(thread.ManagedThreadID, thread);
        }

        thread.Start();
    }

    private void notifyComplete(int threadID)
    {
        bool done = false;
        lock (m_workers)
        {
            m_workers.Remove(threadID);
            done = m_workers.Count == 0;
        }

        if (done) m_completionEvent.Set();
    }
}

Note, I have not tested the code above, so it might not be 100% correct. However I hope it illustrates the concept enough to be useful.