Dynamically Allocate Cache Size to Alleviate HeapSpace Error

Question

We have a collection of objects which grows quite large over time. We have implemented a caching strategy to help alleviate this, however we are still running out of Heap Space at run time - if enough memory isn't allocated at startup.

Is there a standard mechanism to reduce the size of this cache at runtime to remove these OutOFMemory errors? This way if our process is started with a smaller slice of memory than normal we can hopefully avoid the server dying.

I realize that this is an Error type and thus shouldn't be caught / processed as it's normally indicative of more serious issue.

Is it as simple as having something as follows:

private static final long RECOMMENDED_MEMORY = 1073741824L;  //Example 1 Gig
private static int recommendedCacheSize = 100; //Example of 100 items
long heapSize = Runtime.getRuntime().totalMemory();
double size = Math.floor((double) heapSize/RECOMMENDED_MEMORY * recommendedCacheSize);
recommendedCacheSize = ++size; 

Answer 1

I suspect that the caching implementation does not maintain cache of WeakReference or SoftReference objects, which are typically employed in a Java object cache. It is highly important to use soft or weak references for cache, failing which it is possible for an OOME (Out Of Memory Error) to be thrown.

The rational behind using soft/weak references is that the garbage collector will attempt to collect these objects, in case not enough memory is available. Using strong references (the normal object references) to store references to objects in a cache, will prevent memory from being reclaimed.

If however, you get OOMEs even when you're using weak/soft references, I suspect your GC is not well tuned. It appears that your application experiences sudden spurts of memory consumption, resulting in the OOME condition. Such a situation is unlikely, but possible, especially if the previous invocation of the GC did not clear enough memory (which is eventually consumed by the next spurt in consumption).

Answer 2

Why not use something that was designed as a real cache?

While you could use soft references to implement a memory-sensitive cache, such caches tend to leave objects in memory too long (increasing your GC load). Weak references are not suitable for caches, as they have a much higher likelihood of being cleared (in practice, they're cleared on every GC).