views:

101

answers:

3

Does anyone know of a dll to defragment memory for a Windows 2003 server?

Here's the background: We have a .net ecommerce site that uses a pre-made framework for most of the heavy lifting. The website occasionally gets out of memory exceptions when trying to allocate memory when adding objects to the cache. It mostly happens when the framework tries to add large-ish datasets to the cache.

I understand that when you try to add something to the cache it needs to use contingious memory to add the object, if it can't find a big enough chunk of memory, you'll get an out-of memory exception.

The server often gets this problem when IIS is consuming 600MB+ of memory already and needs 10 or 20 MB more. The server has 4GB of memory so IIS should be able to use 2GB, but I think it's these large objects that are having a problem find a place to fit. So my hacky work-around is to maybe try to catch the exception, run a quick RAM defrag, and continue.

I know it would be best to use smaller objects, but I really don't want to have to reverse-engineer the framework to replace parts of the caching code.

Cheers, Lance

+1  A: 

Before you start designing a solution, I would recommend taking some time to better understand the issue. A good starting point is the CLR profiler. It will let you see what's going on with the memory allocations and garbage collector.

It could be that the GC is not able to keep up with the memory allocations. The GC will normally take care of keeping the memory space tidy, including defragmentation. Depending on what you find, you may need to trigger the GC manually.

Kurt
+2  A: 

Firstly, the physical structure of the memory doesn't need defragmenting - the OS manages the pages of physical memory and presents it to the application as a continuous 2GB address space. How this memory is then managed is up to the application (or the CLR in the case of a .NET application)

The GC in most of the CLRs (I believe the .NET 4.0 CLR has made some changes in this area) uses a traditional "linked-list" allocation for the large object heap and it doesn't compact the heap due to the cost of moving large amounts of memory around.

This means that if you are allocating lots of large objects that have varying sizes and varying lifetimes you can easily end up with fragmented memory and out of memory exceptions as you've seen.

I've never seen a memory "defragmenter" DLL as to be able to do this you would have to have full control over the GC and CLR to ensure it updates any references.

Two things worth trying are:

1) if you can clear objects from the cache then when you get an out of memory exception release some of the objects.

2)Try the .NET 4.0 CLR to see if that improves the situation.

Matt Breckon
@Matt - clearing the cache does indeed work to stop the exception from occurring. But of course that means that you've cleared the cache, removing some speed perks of caching.The framework we are using doesn't support .net 4, so that's not an option.I know defragging is a crappy option, but it's the least invasive thing I could think of before I end up rewriting the framework's caching methods!
Lanceomagnifico
As I said, you would have to have full control over the CLR and the garbage collector in order to "defrag" the memory. Can you clear up SOME of the cache (perhaps based on a last used basis?)
Matt Breckon
A: 

This article should have everything you need:

http://msdn.microsoft.com/en-us/magazine/cc163528.aspx For example: "Is Fragmentation a Problem on Your Managed Heap?"

Stu