w3wp crashing out

Question

Hi,

One of our web servers is suffering from random w3wp.exe crashing and after a couple of weeks of debugging i simply cannot figure out why. The only thing that has helped so far is reducing the max worker processes from 15 to 5 however this isn't ideal as we are using a multi-cpu machine in the hopes of reducing the total number of servers needed. We serve a large volume of small requests so parallel processing is a requirement.

As far as I am aware all possible sources of parallel processing collision have been addressed using thread locking.

• Win 2008 64Bit SP2
• IIS7
• Dual 3.1Ghz Xeon
• 4Gb Ram

First error: Application: w3wp.exe Framework Version: v4.0.30319 Description: The process was terminated due to an internal error in the .NET Runtime at IP 70D9CECA (70D40000) with exit code 80131506.

Followed straight away by: Faulting application w3wp.exe, version 7.0.6002.18005, time stamp 0x49e023cf, faulting module clr.dll, version 4.0.30319.1, time stamp 0x4ba1d9ef, exception code 0xc0000005, fault offset 0x0005ceca, process id 0x%9, application start time 0x%10.

Many Thanks

Answer 1

Exception code 0xc0000005 generally points to memory access violation. Look for any unsafe component you may be using.

Answer 2

You are in for a doozy of a ride. These exceptions are extremely tricky to track down and correct.

The first step is to get the IIS Debug Diagnostic Tool (v1.1). Once you have installed this, you'll need to set up some tracking projects and then attach the debugger to your running processes. Keep in mind, this tool collects a LOT of data (it can be in excess of 1GB of stuff), so combing through it may be a hassle, but it has a good potential of telling you what modules are causing the crash and what modules are interfering.

The reason w3wp.exe is crashing, though, is that an unhandle-able exception is occurring during phases of the transaction that your code/health-monitoring/etc are already completed.

In my own personal case, I found that decoupling the session from the process solved the problem. I never discovered the full reason, but the best guess we had was that the memory requirements for paging were too great for the w3wp.exe to handle all at the same time. Once we decoupled into an external session state server, the problem went away.