Making a memory intensive background application "friendly"

Question

I have an application that periodically needs to process large blocks of data with a computationally trivial algorithm. It turns out I can also prevent slowing down the system from hard drive accesses by keeping the blocks of data in a memory cache. The application is a low-priority application so I'm working to minimize its impact on the system across the board, which means using extra memory (when available) to reduce the load on the CPU and hard drives. The cached data is just 64MB blocks of bytes, and the more of them I have in memory the less overhead the program will have on the drives.

What I need to do is dump the in-memory cache whenever any other application on the system needs more physical memory than is available, and do so fast enough that the user never feels the system slowing down due to high memory demands.

I'm particularly interested in how this would be accomplished in a .NET application.

Answer 1

You could P/Invoke down to Win32 CreateMemoryResourceNotification API with the LowMemoryResourceNotification option. This returns a notification object that you can either poll (using QueryMemoryResourceNotification) or wait on (in a background thread; you'd need to use one of the Win32 wait methods rather than a .NET wait method, or possibly derive a custom class from WaitHandle to encapsulate the Win32 object). You would respond by dropping the blocks from your cache and forcing a garbage collection; you'd have to test whether this is "fast enough that the user never feels the system slowing down."

Also, looking at the docs, the threshold for raising a low-memory notification is really low memory (32 MB on a 4 GB system!). I suspect by that time the user might already be feeling the slowdown, and paging your app into memory to dump its blocks might cause perceptible disk access. Again, you could evaluate this with testing.

Answer 2

One option would be to use the ASP.NET Cache, which scavenges items in response to low memory. While Microsoft attaches a big warning that it is only tested within an ASP.NET application, there's nothing to stop you accessing HttpRuntime.Cache in any application, and in practice it has worked when I've done so.

If that feels dirty and wrong (or just doesn't do quite what you need), we can at least draw inspiration from how ASP.NET knows to scavenge the cache. It periodically calls the kernel's GlobalMemoryStatusEx to find out the memory available. Specifically, the dwMemoryLoad property of the returned structure is the percentage of total memory in use. By default, ASP.NET considers memory to be running out when this reaches 90%.

There's some example code to call this yourself here.

Answer 3

You can use the WeakReference class to have weak references to the memory blocks. This allows for the garbage collector to remove the objects if neccessary.

Yor objects would be in the large objects heap, as they are larger than 85 kb. The most common garbage collection only looks at the first generation heap, so it would not collect your objects. If the system notifies the .NET system that it's running out of memory, the garbage collector will run a more thorough collection, removing your objects in the large objects heap.