System.Security.Policy.Evidence, Web Services and Blowing Out the LoH

Question

A new application that has been developed has a heavy use of web services. We started hitting out of memory exceptions on a regular basis (as usage has increased). Upon reviewing the memory dumps I noticed there were a large number of same sized byte[]. Looking at the handles for these byte[] I noticed that they were referenced by System.Security.Policy.Evidence

Upon further review I identified these memory allocations as the actual assemblies (dlls) that had our web service classes in them (2 of the assemblies in particular were in memory 128 times and 115 times). I found some information in here --> blogs.msdn.com/tess/archive/2008/06/25/asp-net-memory-leak-byte-arrays-rooted-in-system-security-policy-evidence.aspx

and here --> blogs.javista.com/2009/03/18/best-practices-for-crm-memory-usage/

but I have not been able to find much other reference to this issue. (.NET framework loading the webservice assembly into memory to check security policys).

Currently, one of the only solutions I am seeing is to seperate the asseblies of the webservices into smaller assemblies that reference libraries.

I am confused why the .NET framework must load the entire assembly into memory to check the policies and wanted to see if anyone else has encountered this and what your solutions were.

Thanks, Dan

Answer 1

I've seen the same thing in memory dumps on Framework 3.5 SP1.

Tess's blog talks about these assemblies being stored in the ASP.NET Cache, and the Cache flushing the assemblies, causing them to be reloaded. The suggestion was that the Cache would flush them when the system was low on memory, which is the default behaviour for anything that goes in the ASP.NET Cache. Tess says that a hotfix resolves this issue.

The current version of System.Web.Services contains this in System.Web.Services.Protocols.ServerProcotol's AddToCache method:

        HttpRuntime.Cache.Insert(this.CreateKey(protocolType, serverType), value, null, Cache.NoAbsoluteExpiration, Cache.NoSlidingExpiration, CacheItemPriority.NotRemovable, null);

Note that this is explicitly setting the cache item to never expire and never be removable. I guess this is what the hotfix changed to resolve the issue -- no more automatic cache flushing.

Now, in our application that is experiencing this problem, we have some code that manually clears everything from the ASP.NET Cache under certain circumstances. I think this is why we are seeing the problem: although the hotfix stopped the ASP.NET Cache clearing these assemblies from the cache automatically, our code is coming along and clearing the cache manually.

I wonder if your application (or any of its third-party components) is doing anything with the Cache which might be deleting these items?