Is implementing a cache considered a difficult problem?

Question

There are many questions on here about caching that doesn't work properly, or asking how to implement caches properly, for all sorts of things from HTTP to SQL queries, L1/L2 memory caching, etc..

Is it generally held to be a difficult problem in computer science terms?

Answer 1

"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton

Answer 2

I think the issue is deciding how you want your cache to operate ? e.g. what strategies do you use for eviction (least recently used etc.). Is it going to be distributed etc.? Do you want to optimise cache hits etc.? Complexity is based around these issues.

Simple in-process caches can be trivial. Generally I've not had to implement these, since you can get off-the-shelf implementations for most platforms, and I would hesitate to reinvent the wheel (I presume your question is about the difficulty of implementation, rather than asking if you should create one yourself)

Answer 3

I don't know if implementing a cache is really considered to be a difficult thing, but I suspect that most people that do don't really test them well enough to be sure that it would work, and that the cache actually does the job it is supposed to do. The thing about implementing a cache is that if you get it wrong, unless you are monitoring cache misses and comparing performance, you won't even know, since the cache failure will just "fail dumb" down to the backing store. So what if the cache has a 100% miss rate, it only makes the default case 1% slower anyway.

Answer 4

I think the difficulty that many programmers have is in restraining themselves from writing caching software for their application. It's good to remember that modern operating systems alrerady do a raft load of caching for you , and that implementing it yourself may well be a pesimisation.

Answer 5

Not until you try it in a nontrivial case.

Answer 6

I think that no matter the task, if there is need for concurrent access to what is difficult is just that: control of concurrent access. Just that can cause serious problems such as dead-lock, serialization in access, limitations on concurrency, among other.