ASP.Net web site gets burst of requests that crashes IIS worker process

Question

I'm running a fairly high volume asp.net web site across a two serve cluster. Typical load is around 20 requests/second per server as logged by the ASP.Net Apps performance object.

At periodic times during the day the performance counter logs anywhere from 2,000 to 9,000 requests/second. The worker process dies at this time and reports a ".Net Runtime 2.0 Error Reporting" application log with "Faulting application w3wp.exe" listed.

What is strange is that IIS doesn't log any burst of requests at this time. The web logs record a normal pace of traffic up until this point, and show no burst of thousands of requests.

The server is running IIS6 on Windows 2003 64 bit Server. It is a quad core xeon with 6GB of RAM.

Only ONE of the two servers in the load balanced cluster record this volume of requests. The other server never shows more than the 20 or so average requests/second.

Any ideas as to why the performance log would show such a high volume of requests that aren't logged anywhere? I am not able to determine what is causing this load, and am wondering if it is an application error of some kind or if it is truly a burst of traffic.

Any suggestions for how to track down the culprit?

Thanks, David

Answer 1

Honestly, it sounds like a potential DoS attack. Can you enable a log on your router/switch to see the raw ethernet frames coming through? Let me know. Thanks.

-Shaun

Answer 2

Get a crashdump of your asp process using Debugging Tools (Windbg & Co).

Check out http://blogs.msdn.com/tess.

Answer 3

After using WinDbg to view a crash dump, it turned out this was caused by a recursive Server.Transfer producing a stack overflow.

Apparently the performance log that shows the the Requests/Sec for ASP.Net Apps v2.0.50727 records a server transfer as a new request. But since it is not doing a client redirect it never shows up in the IIS logs like a DOS attack probably would.

The fact that the log showed high requests/sec made me think it was not an application error.

Thanks for all your suggestions.