Is it advisable to use HashSet as a static global on a webserver?

Question

I want to create a "temporary cache lookup" to speed up a file lookup on my webserver.

What I have now is a folder full of images, and if a user requests an image I use a File.Exists(...) to check to see if the image exists. If it doesn't, download it from another server, and either way, redirect the user to it.

The problem with this is that a lot of requests at once cause File.Exists() to hang. I would like to keep a quick and dirty HashSet of filenames known to be in the local folder, so that if the the user requests a file, and it exists in the HashSet, just redirect to it without doing the File.Exists(), if it doesn't exist int he HashSet, do the File.Exists() then add it.

I know that the HashSet would get blown away if the server ever gets restarted, but I'm not worried about that, because it would quickly "rebuild" itself with the most requested images using the above scenario.

The main question is, since this object would be getting called from multiple users, and various requests would all be adding items to the set, would this cause a problem? Up to now all I've used in the static global sense on webservers is DB connection strings or an email address to send alerts to, etc.

Edit with regards to race conditions:
Yes I was thinking of race conditions. But is a race condition even possible on a single HashSet? Since it only allows unique values, wouldnt the second attempt to add a value fail? At which point I would just ignore the error and continue on.

Answer 1

It makes sense. However, make sure you take care of possible race conditions. You could use the ReaderWriterLockSlim class to control access to the object from different threads.

UPDATE:

You absolutely need to lock the object appropriately, as the Add method is not an atomic operation. You can even leave the object in an inconsistent state if you add two things simultaneously.

Answer 2

In addition to the race condition problems people have mentioned, you are asking for a number of other problems.

First off, a cache without a cache lifetime policy has a name -- we call caches that grow without bound and never release their data "memory leaks". They have a way of bringing servers to their knees. Think hard about how you're going to know when your cache is getting too big and its time to throw it away.

Second, caches are worse than useless if they are liars. Your cache alleges to be a speedup over the file system, but exactly what mechanism do you propose to keep the cache and the file system consistent? Suppose a user accesses a file, the information gets cached, and then some other operation deletes that file. Now the cache is out of sync with the thing is is supposed to be abstracting over; this sounds like a potential source of bugs. The next time someone accesses that file, the cache is going to say that it exists even though it does not any longer.

These are of course solvable problems; the question is not whether they are solvable, but rather, whether the expense of solving them is going to be paid for by the increase in performance you eke out.

Or, you could ignore these problems and hope for the best. For example, if you believe that the cost of running into a memory leak or cache inconsistency is a cost of doing business that the increase in performance pays for. If that's the case then it would be wise to intentionally make the decision to take a brittle low-cost solution, rather than doing so accidentally and being surprised later when things break.