How to cache reading from a file .NET 2.0

Question

I have an httpmodule that reads from a csv file for redirection rules.

However, since it's an httpmodule and it is checking on every request, I obviously don't want to read the file every time.

What are some strategies to cache the rules (in an in Array) so that I do not have to read the file on every request?

Update: Using .NET 2.0

Answer 1

How about a Dictionary? You can just grab the one you want using dict[rule]. Or is it more complicated than that? Convert the csv to XML and use XQueryXPath searches?

Answer 2

The Application storage is the appropriate place to put this assuming that all users will be redirected in the same manner. The Session storage stores information with the context of the current user and is only share for that user. The Application storage functionality keeps the information so that it can be globally shared amongst All users, and guests.

If this information is quite large (>60mb+ or 1000s+ of objects) you might want to consider a serious enterprise level caching component (similar to JCS/EhCache [but for .net])

Answer 3

We do this by storing the rules (or similar) in the Application state.

We load the values in Session_Start (with a test to ensure that we only do it once) - I have a comment in Application_Start that states: "moved to session start because it seems to work better"

Something like this:

Application.Lock();
// Only load the rules if we haven't already
if (Application["RulesCache"] == null)
{
    List<Rule> rules = LoadRules();
    Application["RulesCache"] == rules;
}
Application.Unlock();

We're actually a little more sophisticated than this - but the basic premise is valid.

Pragmatically, you'd want to encapsulate the load into a separate routine and probably provide a mechanism to update the cache without having to restart the application

Answer 4

Use Cache with file dependency

On cache miss you will read your values and also transform them into a list of certain object types and cache them. The benefits:

• unified storage for all users, because they all these values since you're using HttpModule
• automatic memory utilisation management - cache is fast and may get discarded when server is low on memory
• file dependency cache is great when you change the file and your cache will automatically get invalidated and read anew in the first request from the newly changed file so you don't have to restart you application or do some other tricks to refresh cached values
• fast data reads - if you also convert CSV data on file read you gain a lot of time because you will only parse it once so all subsequent data reads will take much less time than if you stored CSV raw data only and parse on every data read

Additional note
It should be noted as well that OS caches files as well. So if you frequently use certain files access will be quicker on consequent reads if they are done in a relative short time-frame. Relative because this depends on server load, memory usage etc. Just from the top of my head. But it would still be much slower than caching described.

Answer 5

There are two ways. Create an Array variable and set it to static.

public class RulesTable
{
    private static readonly RuleCollection _rules;

    public RuleCollection Rules { get; }
}

In this case, you have to create a class like RuleCollection and make it threadsafe.

Or you can use a simple Cache.

System.Web.HttpContext.Current.Cache.Insert("Rules", RulesArray, new System.Web.Caching.CacheDependency("CSV Filename"));