Does WeakReference make a good cache?

views:

904

answers:

+5 Q:

Does WeakReference make a good cache?

Hi,

i have a cache that uses WeakReferences to the cached objects to make them automatically removed from the cache in case of memory pressure. My problem is that the cached objects are collected very soon after they have been stored in the cache. The cache runs in a 64-Bit application and in spite of the case that more than 4gig of memory are still available, all the cached objects are collected (they usually are stored in the G2-heap at that moment). There are no garbage collection induced manually as the process explorer shows.

What methods can i apply to make the objects live a litte longer?

+2 A:

No, WeakReference is not good for that because the behavior of the garbage collector can and will change over time and your cache should not be relying on today's behavior. Also many factors outside of your control could affect memory pressure.

There are many implementations of a cache for .NET. You could find probably a dozen on CodePlex. I guess what you need to add to it is something that looks at the application's current working set to use that as a trigger for purging.

One more note about why your objects are being collected so frequently. The GC is very aggressive at cleaning up Gen0 objects. If your objects are very short-lived (up until the only reference to it is a weak reference) then the GC is doing what it's designed to do by cleaning up as quickly as it can.

Josh Einstein 2009-05-30 17:50:03

"a dozen on CodePlex" ... and one in the Framework. The ASP.NET Cache System.Web.Caching.Cache can be used in non-ASP.NET applications and is quite powerful. Microsoft documentation says it's not recommended to use it in client apps (but doesn't really say why), but I've used it in a wide variety of apps successfully.

Joe 2009-05-30 18:03:17

I tried HttpRuntime.Cache, but it doesn't work as i like. I continously added items to the cache and soon got an OutOfMemoryException instead of having the cache evicting items.

Rauhotz 2009-05-31 08:41:15

"HttpRuntime.Cache, but it doesn't work as i like... OutOfMemoryException". Surprising, but you could try adjusting the cache configuration - e.g. privateBytesLimit and percentagePhysicalMemoryUsedLimit properties. Or there may be some other reason for the OutOfMemoryException.

Joe 2009-06-06 12:39:30

+5 A:

Using WeakReferences as the primary means of referencing cached objects is not really a great idea, because as Josh said, your at the mercy of any future behavioral changes to WeakReference and the GC.

However, if your cache needs any kind of resurrection capability, use of WeakReferences for items that are pending purge is useful. When an item meets eviction criteria, rather than immediately evicting it, you change its reference to a weak reference. If anything requests it before it is GC'ed, you restore its strong reference, and the object can live again. I have found this useful for some caches that have hard to predict hit rate patterns with frequent enough "resurrections" to be beneficial.

If you have predictable hit rate patterns, then I would forgoe the WeakReference option and perform explicit evictions.

jrista 2009-05-30 18:26:14

The problem is that as soon as i have a strong reference the a cached object, i am in danger of running run out of memory before i can set it as collectible.

Rauhotz 2009-05-30 18:36:43

Thats the job of your eviction strategy. Every cache needs two main things...the ability to register an instance in the cache, and a background process that determines which items should be evicted base on a set of rules. There are many existing eviction strategies that may work: LRU (Least Recently Used...i.e. oldest out), LFU (Least Frequently Used...i.e. 10 hits gets dropped in favor of the items with 100 hits), etc. You can't cache every object that gets added forever...you gotta add a background thread to handle eviction.

jrista 2009-05-30 18:41:24

But that background thread will clean the cache only every now and then and eventually too late.

Rauhotz 2009-05-30 22:34:57

Thats up to you. It will clean it up as frequently as you need it yo. Or, if that doesn't work, you can use thread signaling to have it clean up on a lazy schedule, plus on demand. For example, you could use a ManualResetEvent to have your main cache code signal its background worker that it should wake up immediately and purge, reset the event, and go back to sleep. And you have me wondering now what exactly your caching and how much of it. If you are seriously worried that your CACHE, of all things, is going to consume more memory than a 64bit machine offers...it sounds like you have other

jrista 2009-05-30 22:57:21

problems. You need to consider that what your caching and the volume of items that your caching really, truely do need to be cached, because if your consuming every available scrap of memory your process has on a 64bit system, something is gravely wrong.

jrista 2009-05-30 22:58:15

It's quite simple: The cache should use all free (physical) memory that is available, but as soon as another process or another method in the same process is requiring the memory, it should remove old items.

Rauhotz 2009-05-31 08:52:40

I am honestly not sure how you could accomplish that. Your "cache" would need to not only be aware of its own memory usage, but also that of the entire program, AND external, unrelated processes. Using WeakReferences as the dominant form of reference in your cache is not going to work, because your objects are going to get collected as soon as possible, defeating the purpose of having the cache. You are actually going to need to hook into the system and watch for messages regarding memory requests...and that is going to severely impact the performance of your cache...rendering it useless.

jrista 2009-05-31 09:07:50

Just out of curiosity...why do you need to consume every scrap of memory? Seems highly inefficient...can you give us a more high-level overview of what your trying to do that would require such a massive cache?

jrista 2009-05-31 09:08:44

Unused memory is useless memory, so why not using it? The cache should store a few large tables with each several hundreds of MB that are expensive to fetch, so i want to keep as many of them in memory if it is available, but not take the memory take it away from the rest of the system.

Rauhotz 2009-05-31 09:42:01

Well, assuming you actually need to cache hundreds of megs of database tables, then I guess you do what you gotta do. Just make sure that your not prematurely optimizing a performance issue that may never exist. Database servers these days are incredibly efficient applications, and handle caching of data pages about as optimally as is possible. Unless your writing a database yourself, I would say let the database do what it does best, and put effort into making your application do something useful before you worry about filling "wasted" memory just for the sake of filling it.

jrista 2009-05-31 09:47:59

The database may be fast, but the network may be not. But even if the database runs on the same machine as the application server, it takes some time to load a table with some million rows into the memory of the application server, which proofed to be the bottleneck in our scenario. Having it already there is a nice thing.

Rauhotz 2009-05-31 10:01:54

+1 A:

In .net, a WeakReference is not considered a reference from the GC standpoint at all, so any object that only has weak references will be collected in the next GC run (for the appropriate generation).

That makes weak reference completely inappropriate for caching - as your experience shows.

You need a "real" cache component, and the most important thing about caching is to get one where the eviction policy (that is, the rules about when to drop an object from the cache) are a good match for you application's usage pattern.

Nir 2009-05-30 19:57:57

ansaurus

tags:

views:

answers:

Does WeakReference make a good cache?

related questions