What's the best way to pass data between concurrent threads in .NET?

Question

I have two threads, one needs to poll a bunch of separate static resources looking for updates. The other one needs to get the data and store it in the database. How can thread 1 tell thread 2 that there is something to process?

Answer 1

I use Monitor.Wait / Pulse on a Queue of work items.

Answer 2

If the pieces of data are independant then treat the pieces of data as work items to be processed by a pool of threads. Use the thread pool and QueueUserWorkItem to post the data to the thread(s). You should get better scalability using a pool of symmetric threads and limiting the amount of synchronisation that has to occur between the producer and consumer(s).

For example (from MSDN):

    TaskInfo ti = new TaskInfo("This report displays the number {0}.", 42);

    // Queue the task and data.
    if (ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadProc), ti)) {    
        Console.WriteLine("Main thread does some work, then sleeps.");

        // If you comment out the Sleep, the main thread exits before
        // the ThreadPool task has a chance to run.  ThreadPool uses 
        // background threads, which do not keep the application 
        // running.  (This is a simple example of a race condition.)
        Thread.Sleep(1000);

        Console.WriteLine("Main thread exits.");
    }
    else {
        Console.WriteLine("Unable to queue ThreadPool request."); 
    }


// The thread procedure performs the independent task, in this case
// formatting and printing a very simple report.
//
static void ThreadProc(Object stateInfo) {
    TaskInfo ti = (TaskInfo) stateInfo;
    Console.WriteLine(ti.Boilerplate, ti.Value); 
}

Answer 3

Does the "store in the DB" thread always need to be running? It seems like perhaps the best option (if possible) would be to have the polling thread spin up another thread to do the save. Depending on the number of threads being created though, it could be that having the first polling thread use ThreadPool.QueueUserWorkItem() might be the more efficient route.

For more efficiency, when saving to the database, I would use async I/O on the DB rather than the sync methods.

Anytime you can get away from having to communicate directly between two threads, you should. Having to throw together some sync primitives, your code won't be as easy to debug and could introduce some very subtle race conditions that cause "once in a million execution" type bugs (which are far from fun to find/fix).

If the second thread always needs to be executing, let us know why with some more information and we can come back with a more in-depth answer.

Good luck!

Answer 4

I personally would have thread 1 raise events which thread 2 can respond to. The threads can be wired up to the appropriate events by the controlling process which initiates both the threads.