Using a queue in a multithreaded situation in C#

Question

I just learned about queues in .NET and I have a few questions.

Let's say that I'm building an application that downloads the HTML of pages and then processes it. Here's how I want it to operate:

1. A main thread adds URLs to a queue
2. This queue is read by two other threads. They "dequeue" a URL and then download the corresponding HTML.
3. The HTML is then sent back to the main thread.
4. When the HTML arrives in the main thread, it is placed into another queue, handled by another two threads. These threads process the HTML.
5. The results of the processing are returned to the main thread.

---

How can I implement such a scenario without the possibility of a race condition?

Also, what is the best way to pass the information between queues and threads as described above?

Could you give me some sample code?

Thanks!

Answer 1

I recommend BlockingCollection<T>. It represents a "producer/consumer queue" common in multithreading.

Answer 2

There are two ways that come to mind.

The first is to use a collection that implements locking to produce thread safe consistency (such as BlockingCollection as mentioned elsewhere).

The second is to use an immutable collection (such as the ones described by Eric Lippart in http://blogs.msdn.com/b/ericlippert/archive/2007/11/13/immutability-in-c-part-one-kinds-of-immutability.aspx ) which avoids most of the hassle of thread locking and race conditions in the first place and makes what's ever left over a little more obvious usually.