C# Spawn Multiple Threads for work then wait until all finished

Question

just want some advice on "best practice" regarding multi-threading tasks.

as an example, we have a C# application that upon startup reads data from various "type" table in our database and stores the information in a collection which we pass around the application. this prevents us from hitting the database each time this information is required.

at the moment the application is reading data from 10 tables synchronously. i would really like to have the application read from each table in a different thread all running in parallel. the application would wait for all the threads to complete before continuing with the startup of the application.

i have looked into BackGroundWorker but just want some advice on accomplishing the above.

1. Does the method sound logical in order to speed up the startup time of our application
2. How can we best handle all the threads keeping in mind that each thread's work is independent of one another, we just need to wait for all the threads to complete before continuing.

i look forward to some answers

Answer 1

If you're not on .NET 4.0 then you can use a List<ManualResetEvent>, one for each thread and Wait for them to be Set. To wait on multiple threads you could consider using WaitAll but watch out for the limit of 64 wait handles. If you need more than this, you can just loop over them and wait for each one individually.

If you want a faster startup exprience, you probably don't need to wait for all the data to be read during startup. Just display the GUI and any information that is missing can be shown greyed out with some sort of "Updating..." icon or similar. When the information comes in, just fire an event to update the GUI. There could be many operations that the user can begin to perform even before all the data from all tables is read in.

Answer 2

If you're feeling adventurous you can use C# 4.0 and the Task Parallel Library:

Parallel.ForEach(jobList, curJob => {
  curJob.Process()
});

Answer 3

Assuming the database reader threads return as soon as they're done, you can simply call Thread.Join on all ten threads in turn from the initiating thread.

Answer 4

If you have more than 64 wait handles for an STA Thread as Mark says. you could create a list with your threads and wait for all to complete in a second loop.

//check that all threads have completed.
foreach (Thread thread in threadList)
{
     thread.Join();

}  

Answer 5

Just for fun, what @Reed has done, with Monitor. :P

class Program
{
    static void Main(string[] args)
    {
        int numThreads = 10;
        int toProcess = numThreads;
        object syncRoot = new object();

        // Start workers.
        for (int i = 0; i < numThreads; i++)
        {
            new Thread(delegate()
            {
                Console.WriteLine(Thread.CurrentThread.ManagedThreadId);
                // If we're the last thread, signal
                if (Interlocked.Decrement(ref toProcess) == 0)
                {
                    lock (syncRoot)
                    {
                        Monitor.Pulse(syncRoot);
                    }
                }
            }).Start();
        }

        // Wait for workers.
        lock (syncRoot)
        {
            if (toProcess > 0)
            {
                Monitor.Wait(syncRoot);
            }
        }

        Console.WriteLine("Finished.");
    }
}

Answer 6

My preference for this is to handle this via a single WaitHandle, and use Interlocked to avoid locking on a counter:

class Program
{
    static void Main(string[] args)
    {
        int numThreads = 10;
        ManualResetEvent resetEvent = new ManualResetEvent(false);
        int toProcess = numThreads;

        // Start workers.
        for (int i = 0; i < numThreads; i++)
        {
            new Thread(delegate()
            {
                Console.WriteLine(Thread.CurrentThread.ManagedThreadId);
                // If we're the last thread, signal
                if (Interlocked.Decrement(ref toProcess) == 0)
                    resetEvent.Set();
            }).Start();
        }

        // Wait for workers.
        resetEvent.WaitOne();
        Console.WriteLine("Finished.");
    }
}

This works well, and scales to any number of threads processing, without introducing locking.

Answer 7

If you are using .NET 3.5 or below, you can use an array of AsyncResult or BackgroundWorker and count how many threads have returned (just don't forget to decrease counter with interlocked operations) (see http://www.albahari.com/threading/ as a reference).

If you are using .NET 4.0 a parallel for is the simplest approach.

Answer 8

I like @Reed's solution. Another way to accomplish the same in .NET 4.0 would be to use a CountdownEvent.

class Program
{
    static void Main(string[] args)
    {
        var numThreads = 10;
        var countdownEvent = new CountdownEvent(numThreads);

        // Start workers.
        for (var i = 0; i < numThreads; i++)
        {
            new Thread(delegate()
            {
                Console.WriteLine(Thread.CurrentThread.ManagedThreadId);
                // Signal the CountdownEvent.
                countdownEvent.Signal();
            }).Start();
        }

        // Wait for workers.
        countdownEvent.Wait();
        Console.WriteLine("Finished.");
    }
}