Custom multithreading loop

Question

Hi,

I am looking into using a producer\consumer threading pattern to run through a list of strings that i retrieve from a database and then am going to run powershell commands against. As the powershell commands can be quite intensive i only want to spawn 3 threads at a time and then wait for one thread to finish before doing anymore. I looked into the treadpool but it looks like it is used for more lightweight processing.

I am new to threading and have seen a few producer\consumer examples on the internet but am not sure how to handle the 3 thread loop situation using waithandles (if thats the way to do it). I think i have to use the waithandle.waitany to achieve the above but am not sure how to implement this. If someone could point me in the right direction i would be most appreciative

Thanks

Answer 1

Use a queue where you add the work items. Then spawn as many threads as you want to run concurrently, and have them run a loop, trying to get items from the work item queue (make sure that this is happening in a properly synchronized fashion).

If the queue is filled before you start processing, you can use the empty queue as condition to end the loop in the thread. In this case you can wait for all threads to stop so you know when all items have finished processing.

Answer 2

If you could use .NET 4.0, you could investigate using the Task class and a TaskScheduler instance with a limited MaximumConcurrencyLevel set.

Answer 3

Check the the CCR/DSS libraries for .Net:
http://msdn.microsoft.com/en-us/library/bb648752.aspx
http://www.microsoft.com/ccrdss/