Should I cache data pre-emptively

Question

I have an application that receives messages from devices every few minutes. I also have clients that request the last 10 messages for a particular device.

I am suffering with some database saturation and I wish to cache this list by device. The basic premise is that when a message is received from the device then the processor that receives the messages will invalidate the cache for that device.

My question is whether I should just invalidate the cache and then have it rebuilt when the next client connects, or should I have the device processor rebuild the cache pre-emptively. The device processor can retrieve the current cache pop the last entry off, add the new entry and cache the new result.

I appreciate that this may be an it depends, answer but I would appreciate hearing peoples own experiences in this area.

Answer 1

I hear ya, but it really does depend. On many, many variables.

If it were me, I would probably just invalidate the cache and let the next client rebuild it since that seems slightly simpler, but I don't see how you could possibly determine which way works better without trying them both.

Hopefully, you can come up with a way to realistically simulate a heavy client load so you're not messing around with the live server.

Answer 2

I think you're describing a "pre-fetch" mechanism, just to help you put a name on it. :)

I don't have a great deal of experience in this particular area, but if you believe you can pre-fetch data and reliably predict that it's what the client wants, and you obtain a measurable and desirable performance improvement because of it, then go ahead and give it a whirl.

Just remember to keep all the hairiness of caching in mind. How is it invalidated when the underlying data changes, etc. Good luck!

Answer 3

I don't think you can answer this question until you've quantified how many times a client on average retrieves messages for a given device. If a given device is only queried for messages once in a blue moon, then it's fine to purge the message cache on each client request. However, if a given device's message queue is queried many times, then a preemptive caching on device sync is likely the best option; given the device syncs are less frequent then the client requests.

Your best might be writing a system that adaptively caches based on load. If a given device's message queue is queried often, it refreshes the cache on device sync. If a device message queue is rarely queried, you refresh the cache on client request.