ASP.NET processing abruptly stopped on IIS 6.0

Question

Hi there

we noticed a rather strange behavior on one of our web applications. When debugging it on a local developer workstation, everything is processed fine (using Cassini Developement Webserver). When published on a remote webserver using IIS 6.0, processing runs fine about 4 or 5 minutes, then abruptly dies.

I'll explain later what "abruptly dies" means, first i would like to give an excerpt of the failing code.

// loads data from SAP Webservice and serializes it into database    
LoadXMLDataFromSAP();

// loop each item of a certain structure and parse data
foreach (var xItem in xSapData)
{
    // method determining a status, about 30 LOC, fast execution
    GetStatusCodeForContract(xItem);

    ...

    // methods to parse data blocks, about 400 LOC, slow execution (database etc.)
    TimeconsumingParserMethod1(xItem);
    TimeconsumingParserMethod2(xItem);
}

This code is running fine when Debugging, execution takes about 13 minutes (thats ok, because it only occurs when synchronizing all data with SAP). When running the SAME code against the SAME SAP-datasource on IIS 6.0, execution stops after 4 or 5 minutes.

First, the program throws a System.NullReferenceException on the call "TimeconsumingParserMethod1()", in the next loop we get the System.NullReferenceException on "GetStatusCodeForContract()". As we use the same parameter on both calls, i think the NullReferenceException is thrown because the member "xItem" is null.

Server configuration:

8 Core Intel machine
4 GB of RAM
RequestTimeout = 900 (15 minutes)
Memory usage = unlimited

Does anybody know about such a behavior on IIS 6 or similar production server environments?

Answer 1

Well, i finally tracked down the problem. Due some issues with the EventLog on our local testserver, i didn't notice some of the IIS Exceptions.

The root of the problem wasn't any timeout-setting, its the AppDomain recycling behavoir of ASP.NET 2.0. If you change alot of folder structures during a webrequest, the AppDomain gets reloaded which leads to the mysterious NullReferenceExceptions.

The following article helped me track down the issue: http://weblogs.asp.net/owscott/archive/2006/02/21/ASP.NET-v2.0-%5F2D00%5F-AppDomain-recycles%5F2C00%5F-more-common-than-before.aspx

The code snippet below the link is the workaround used (it just disables file monitoring for subfolders, changes to web.config and assemblys still reload the AppDomain for auto-deployment).

System.Reflection.PropertyInfo p = typeof(System.Web.HttpRuntime).GetProperty("FileChangesMonitor", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Public | System.Reflection.BindingFlags.Static);

object o = p.GetValue(null, null);

System.Reflection.FieldInfo f = o.GetType().GetField("_dirMonSubdirs", System.Reflection.BindingFlags.Instance | System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.IgnoreCase);

object monitor = f.GetValue(o);

System.Reflection.MethodInfo m = monitor.GetType().GetMethod("StopMonitoring", System.Reflection.BindingFlags.Instance | System.Reflection.BindingFlags.NonPublic); m.Invoke(monitor, new object[] { });