If I use the ThreadPool in a nested way, my application hangs:
ThreadPool.QueueUserWorkItem((state) =>
ThreadPool.QueueUserWorkItem(Action));
How to get a second and independent ThreadPool to achieve nesting?
views:
295answers:
4Well, technically, there's one per *process* (or one per AppDomain, I'm not sure), so you *could* get multiples involved. It just seems like a really bad idea. :)
technophile
2010-02-05 20:13:17
True. This particular implementation appeared to be confined to a single process.
Joe
2010-02-05 21:59:08
+2
A:
Your application is probably hanging because you are filling it up to the point where all of the active threads are waiting on queued threads. (I assume the original thread is waiting for the work item(s) it queues to complete.)
The ThreadPool has by default 25 * (number of CPUs) worker threads (IIRC).
You probably want to rework the way you're queuing items. If you're queuing them up and then finishing and exiting the thread, you're fine, but having work items hanging around waiting on other processes is generally a bad design; if that's what you're doing you probably need to use real threads or a completely different design. Using real threads is probably an equally bad idea because all that will do (based on what I can surmise of your usage) you'll just create a large number of threads, which will do nothing good for your performance.
A better design might be to have some kind of queue or stack that a few (2-4 depending on number of CPUs) worker threads add items to and pop items off to work. If an item needs to queue a new item it just adds it to the stack (and adds itself back to the stack with some kind of dependency tracking if it needs to wait for the other item).
technophile
2010-02-05 20:12:45
It would be interesting to get a definitive number of threads per threadpool. I've seen it somewhere, just can't recall where. Anyway, I would create a new threadpool rather than trying to manhandle the threadpool into a queue - just keep a collection of the instanced threadpools (array, list, dict).
dboarman
2010-02-05 20:20:13
@dboarman you can actually set the threadpool size using the static methods on ThreadPool
Rob Fonseca-Ensor
2010-02-05 20:21:46
I agree. You should look at your worker threads to see if there's shared objects that could cause a deadlock, and revisit your architecture to realign your thread "boundaries."
ebpower
2010-02-05 20:22:30
@dboarman I'm not suggesting "manhandling". If he's got tasks that need to execute other tasks recursively, then a stack or queue (depending on execution semantics) is probably the cleanest way to handle it. Allowing a thread explosion (threads creating threads creating threads) is just going to cause performance issues, whether it's in a threadpool or via System.Threading.Threads.
technophile
2010-02-05 20:27:17
@technophile: sorry, I wasn't clear - my comment was directed towards the OP on manhandling. It seems like nesting them would be the worst kind of coercion.
dboarman
2010-02-05 21:22:40
@Rob: I know you can increase the size of the ThreadPool - honestly I was wondering why the OP doesn't simply do that. If a new threadpool is the only way to get an implementation solved, then so be it.
dboarman
2010-02-05 21:25:14
Increasing the size of the threadpool is more likely to (usually only slightly) delay the problem than solve it, in my experience.
technophile
2010-02-05 21:27:59
@dboarman @technophile i'm with technophile. There must be a way to do this that isn't deadlock-prone instead
Rob Fonseca-Ensor
2010-02-06 09:50:23
roger roger, first thing would to not nest threadpools... if .Net 4.0 is not an option (which is quite likely), the OP should start out with only a handful (say 5) threads. If more are needed then spin one up - up to say a max of 10. The OP can still use the ThreadPool.QueueUserWorkItem when one of the threads has gone idle. Does this sound right?
dboarman
2010-02-06 10:25:32
I would probably just spin up say 2xCPUs number of WorkItem threads and have them never exit, and then use one or more queues/stacks to manage their workload. That assumes workload is pretty steady; if this is a situation where the need for background tasks is rare or unpredictable you might do it differently. OTOH, if this is the major purpose of the app, though, I might look at using something like Windows Workflow Foundation instead.
technophile
2010-02-07 04:09:48
+2
A:
There is only one single ThreadPool - it's not something you can (or should) make more than one instance of in an application.
I don't recommend doing this, but if you really wanted to, you could use multiple instances of your own ThreadPool implementation, such as SmartThreadPool. This would, technically, allow separate "thread pools".
However, I suspect you're hanging due to a deadlock, not due to the ThreadPool usage. I would investigate where you're getting the hangs. The VS2010 concurrency visualizer is very nice for this, if you have a copy of the VS 2010 beta installed.
Reed Copsey
2010-02-05 20:17:04
+4
A:
You are using the wrong tools.
In .NET 4.0 they introduced the Task Parallel Library. This allows you to do things like use multiple thead pools as well as have parent-child relationships between work items.
Start with the Task class, which replaces ThreadPool.QueueUserWorkItem.
http://msdn.microsoft.com/en-us/library/system.threading.tasks.task(VS.100).aspx
EDIT
Example of creating your own thread pool using Task and TaskScheduler.
http://msdn.microsoft.com/en-us/library/dd997413(VS.100).aspx
Jonathan Allen
2010-02-05 20:20:01
This is not technically correct. The TPL is built upon the .NET ThreadPool - it does NOT create multiple thread pools, but rather, uses the (improved) ThreadPool in the framework for scheduling of tasks.
Reed Copsey
2010-02-05 20:31:32
The TaskScheduler in Task.Start(TaskScheduler) is abstract, so you can construct your thread pools anyway you want.
Jonathan Allen
2010-02-05 22:12:04