views:

354

answers:

3

While my .Net 3.5 app was running, the Windows Task Manager shown that my app had 16 threads. I collected a memory dump for the process and opened it using WinDbg/SOS.

Running the !threads command reveals that I have :

ThreadCount: 456
UnstartedThread: 0
BackgroundThread: 6
PendingThread: 0
DeadThread: 449
Hosted Runtime: no

Here are the first few lines of the !threads output:

       ID OSID ThreadOBJ    State     GC       Context       Domain   Count APT Exception
   0    1 2848 004366a8      6020 Enabled  11738178:11738778 0042a9f0     0 STA
   2    2 1820 004430e0      b220 Enabled  00000000:00000000 0042a9f0     0 MTA (Finalizer)
   7    5 2c38 055d6330    80a220 Enabled  00000000:00000000 0042a9f0     0 MTA (Threadpool Completion Port)
   8    4  e18 04116900   180b220 Enabled  1157cdc8:1157e778 0042a9f0     0 MTA (Threadpool Worker)
XXXX    6    0 055f94b0      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX    7    0 05649228      9820 Enabled  00000000:00000000 0042a9f0     0 MTA
XXXX    8    0 0567d4f8      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX    9    0 05688d68      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX    a    0 056fd680      9820 Enabled  00000000:00000000 0042a9f0     0 MTA
XXXX    b    0 0575d7f0      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX    c    0 056fd250      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX    d    0 0572a780      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX    e    0 0f082668      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX    f    0 0f082a38      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX   10    0 0570ca68      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX   11    0 0570ce50      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
  10   12 3fb0 0570d238   180b220 Enabled  00000000:00000000 0042a9f0     0 MTA (Threadpool Worker)
XXXX   13    0 0570d620      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX   14    0 0570da08      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX   15    0 0570ddf0      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX   16    0 0570e1d8      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX   17    0 0570e5c0      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX   18    0 0579e540      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX   19    0 0579e928      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX   1a    0 0579ed10      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX   1b    0 0579f0f8      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX   1c    0 0579f4e0      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX   1d    0 0579f8c8      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX   1e    0 0579fcb0      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX   1f    0 057a0098      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn
XXXX   20    0 057a0480      9820 Enabled  00000000:00000000 0042a9f0     0 Ukn

If I attach the Visual Studio Debugger to the running process, the Threads windows shows 7 threads.

I have a few questions:

  • Why does WinDbg say there are 456 threads and Task Manager says 16
  • Why does Task Manager say there are 16 threads and Visual Studio Debugger says 7
  • The !thread command shows that all those dead threads don't have a OSID value. Does that mean they are no longer known to the OS are and just .Net objects that are layout around? What is a dead thread exactly?
  • Should I worry about the high number of dead threads?

EDIT Feb 11th 2010: Here is more information about my app. We use background threads to poll the server and perform other tasks. Those tasks are executed every few minutes. We do not use the .Net Thread Pool.

EDIT Feb 18th 2010: I fixed the managed Thread object leak in our program (thanks to @highphilosopher). However, my question about why WinDbg, Task Manager and VS Debugger don't agree on the number of threads is still unanswered. Can anyone explain?

EDIT Mar 1st 2010: I'm still interested in knowing why Task Manager and Visual Studio Debugger don't agree on the number of threads. Why does Visual studio filter some of the threads? What kind of threads does it filter out?

+1  A: 

My guess is the thread pool had periods of heavy load and then killed the threads when the load died down.

ChaosPandion
When I collected the memory dump, my app had been idle for a long time (hours).
Sly
+1  A: 

What you're seeing sounds like you have a reference to these threads on a different thread. Since the thread is referenced it can't be Collected by the GC. The thread has completed execution of the method that it was sent to do so it is in neither a sleep state, nor runnable state, so it must be dead. Check your code for a Collection of threads or something similar. Perhaps an event that gets hooked to by the thread, but never unhooked?

highphilosopher
You're right, we are leaking managed Thread objects. I confirmed that using .Net Memory Profiler. Still, I'd like to understand why Visual Studio and the Task Manager don't agree on the number of threads...
Sly
This may not be the answer that you wanted, but task manager reports some rather odd values sometimes. One good example is with memory. The memory counter in Task manager is you're application's working set. This is pretty much useless information because part of this memory is shared with other processes (depending on your app code, managed, or unmanaged, etc.). I try to steer clear of task manager for numbers, and stick to perfmon whenever possible.
highphilosopher
+7  A: 

Task manager reports the total number of threads for your process while !threads reports the number of managed threads. If you use the ~ command in WinDbg you will see all thread threads for the process.

The output from !threads in your output shows a lot of dead threads. The threads listed with XXXX for id are threads that have terminated but the corresponding thread objects have not yet been collected. I.e. the reported number is much higher than the actual number of threads. The thread count numbers state that 449 of the 456 threads are dead.

I find the number of threads high and if the application has been idle it is odd that the're still around, but without further info it is hard to be more specific.

Brian Rasmussen
@Brian : Thanks! How about the Visual Studio Debugger? Why does it report 7 threads and not 16 as Task Manager does? I expected that VS would either report 456 threads or 16.
Sly
@Sly: Sorry I didn't address the VS question. I reckoned it was a typo since you were talking about WinDbg. My guess is that VS filters some of the CLR threads from the display.
Brian Rasmussen
If I attach to a process while it is being debugged by VS, I see more threads in WinDbg using the ~ command than VS displays, so VS seems to filter some of them.
Brian Rasmussen