w3wp.exe in ASP.NET production app is using 100% CPU. How to find the problem ?

Question

Hi,

we have an asp.net app in production where w3wp.exe is taking 100% CPU ( 4 cores - 4 threads at 25% ) and cpu load never goes down until we recycle the application pool ( the app is alone in the application pool ). Our error log has nothing, there is no exceptions being emitted ( or at least we don't catch them ) so we suspect it's code problem ( infinite loop / deadlock ). The problem only arises after some hours running with high load ( several thousand users ).

There is any way to profile one of the EXISTING threads that is causing cpu load ? After taking a look to JetBrains's DotTrace Profiler seems like it's not possible for limitations of Profiling API and man, we haven't get to reproduce the problem in our testing environment. The app uses SQL Server 2005, LINQ2SQL and System.Transactions API.

Any suggestion to find the problem ?

Answer 1

Create a memory dump of the currently running application and then download WinDbg and SoS (Son of Strike) take a look at the memory profile of the running application to see what is consuming the most memory, where the performance issues are. If you have a .NET 4 application, you can load the memory dump in Visual Studio 2010 and get more of a visual look at what is consuming memory. You'll be able to see wha processes are currently running an be able to gauge these against CPU usage.

The dnr tv episode - Tess Ferrandez on Debugging in .NET is a good introduction on how to use WinDbg for debugging an ASP.NET application.

Answer 2

Its time to get to grips with windbg. I would suggest working through Tess Ferrandez's debugging labs to get a fast start on creating memory dumps and analyzing them in windbg.

http://blogs.msdn.com/b/tess/archive/tags/debugging+labs/

Basic windbg skills will often get you to the root cause of these types of issues faster than any other method. Especially useful where you are unable to reproduce a problem outside of the live environment