Caching best practices - Single object or multiple entries?

Question

Does anyone have any advice on which method is better when caching data in a C# ASP.net application?

I am currently using a combination of two approaches, with some data (List, dictionaries, the usual domain-specific information) being put directly into the cache and boxed when needed, and some data being kept inside a globaldata class, and retrieved through that class (i.e. the GlobalData class is cached, and it's properties are the actual data).

Is either approach preferable?

I get the feeling that caching each item separately would be more sensible from a concurrency point of view, however it creates a lot more work in the long run with more functions that purely deal with getting data out of a cache location in a Utility class.

Suggestions would be appreciated.

Answer 1

How about the best (worst?) of both worlds?

Have the globaldata class manage all the cache access internally. The rest of your code can then just use globaldata, meaning that it doesn't need to be cache-aware at all.

You could change the cache implementation as/when you like just by updating globaldata, and the rest of your code won't know or care what's going on inside.

Answer 2

Under what conditions do you need to invalidate your cache? Objects should be stored so that when they are invalidated repopulating the cache only requires re-caching the items that were invalidated.

For example if you have cached say a Customer object that contains the delivery details for an order along with the shopping basket. Invalidating the shopping basket because they added or removed an item would also require repopulating the delivery details unnecessarily.

(NOTE: This is an obteuse example and I'm not advocating this just trying to demonstrate the principle and my imagination is a bit off today).

Answer 3

There's much more than that to consider when architecting your caching strategy. Think of your cache store as if it were your in-memory db. So carefully handle dependencies and expiration policy for each and every type stored in there. It really doesn't matter what you use for caching (system.web, other commercial solution, rolling your own...).

I'd try to centralize it though and also use some sort of a plugable architecture. Make your data consumers access it through a common API (an abstract cache that exposes it) and plug your caching layer at runtime (let's say asp.net cache).

You should really take a top down approach when caching data to avoid any kind of data integrity problems (proper dependecies like I said) and then take care of providing synchronization.

Answer 4

You can see Microsoft Best Practices in the Caching application block of Enterprise library.

You can read an introduction here

Answer 5

Generally the cache's performance is so much better than the underlying source (e.g. a DB) that the performance of the cache is not a problem. The main goal is rather to get as high cache-hit ratio as possible (unless you are developing at really large scale because then it pays off to optimize the cache as well).

To achieve this I usually try to make it as straight forward as possible for the developer to use cache (so that we don't miss any chances of cache-hits just because the developer is too lazy to use the cache). In some projects we've use a modified version of a CacheHandler available in Microsoft's Enterprise Library.

With CacheHandler (which uses Policy Injection) you can easily make a method "cacheable" by just adding an attribute to it. For instance this:

[CacheHandler(0, 30, 0)]
public Object GetData(Object input)
{
}

would make all calls to that method cached for 30 minutes. All invocations gets a unique cache-key based on the input data and method name so if you call the method twice with different input it doesn't get cached but if you call it >1 times within the timout interval with the same input then the method only gets executed once.

Our modified version looks like this:

using System;
using System.Diagnostics;
using System.IO;
using System.Reflection;
using System.Runtime.Remoting.Contexts;
using System.Text;
using System.Web;
using System.Web.Caching;
using System.Web.UI;
using Microsoft.Practices.EnterpriseLibrary.Common.Configuration;
using Microsoft.Practices.Unity.InterceptionExtension;


namespace Middleware.Cache
{
    /// <summary>
    /// An <see cref="ICallHandler"/> that implements caching of the return values of
    /// methods. This handler stores the return value in the ASP.NET cache or the Items object of the current request.
    /// </summary>
    [ConfigurationElementType(typeof (CacheHandler)), Synchronization]
    public class CacheHandler : ICallHandler
    {
        /// <summary>
        /// The default expiration time for the cached entries: 5 minutes
        /// </summary>
        public static readonly TimeSpan DefaultExpirationTime = new TimeSpan(0, 5, 0);

        private readonly object cachedData;

        private readonly DefaultCacheKeyGenerator keyGenerator;
        private readonly bool storeOnlyForThisRequest = true;
        private TimeSpan expirationTime;
        private GetNextHandlerDelegate getNext;
        private IMethodInvocation input;


        public CacheHandler(TimeSpan expirationTime, bool storeOnlyForThisRequest)
        {
            keyGenerator = new DefaultCacheKeyGenerator();
            this.expirationTime = expirationTime;
            this.storeOnlyForThisRequest = storeOnlyForThisRequest;
        }

        /// <summary>
        /// This constructor is used when we wrap cached data in a CacheHandler so that 
        /// we can reload the object after it has been removed from the cache.
        /// </summary>
        /// <param name="expirationTime"></param>
        /// <param name="storeOnlyForThisRequest"></param>
        /// <param name="input"></param>
        /// <param name="getNext"></param>
        /// <param name="cachedData"></param>
        public CacheHandler(TimeSpan expirationTime, bool storeOnlyForThisRequest,
                            IMethodInvocation input, GetNextHandlerDelegate getNext,
                            object cachedData)
            : this(expirationTime, storeOnlyForThisRequest)
        {
            this.input = input;
            this.getNext = getNext;
            this.cachedData = cachedData;
        }


        /// <summary>
        /// Gets or sets the expiration time for cache data.
        /// </summary>
        /// <value>The expiration time.</value>
        public TimeSpan ExpirationTime
        {
            get { return expirationTime; }
            set { expirationTime = value; }
        }

        #region ICallHandler Members

        /// <summary>
        /// Implements the caching behavior of this handler.
        /// </summary>
        /// <param name="input"><see cref="IMethodInvocation"/> object describing the current call.</param>
        /// <param name="getNext">delegate used to get the next handler in the current pipeline.</param>
        /// <returns>Return value from target method, or cached result if previous inputs have been seen.</returns>
        public IMethodReturn Invoke(IMethodInvocation input, GetNextHandlerDelegate getNext)
        {
            lock (input.MethodBase)
            {
                this.input = input;
                this.getNext = getNext;

                return loadUsingCache();
            }
        }

        public int Order
        {
            get { return 0; }
            set { }
        }

        #endregion

        private IMethodReturn loadUsingCache()
        {
            //We need to synchronize calls to the CacheHandler on method level
            //to prevent duplicate calls to methods that could be cached.
            lock (input.MethodBase)
            {
                if (TargetMethodReturnsVoid(input) || HttpContext.Current == null)
                {
                    return getNext()(input, getNext);
                }

                var inputs = new object[input.Inputs.Count];
                for (int i = 0; i < inputs.Length; ++i)
                {
                    inputs[i] = input.Inputs[i];
                }

                string cacheKey = keyGenerator.CreateCacheKey(input.MethodBase, inputs);
                object cachedResult = getCachedResult(cacheKey);

                if (cachedResult == null)
                {
                    var stopWatch = Stopwatch.StartNew();
                    var realReturn = getNext()(input, getNext);
                    stopWatch.Stop();
                    if (realReturn.Exception == null && realReturn.ReturnValue != null)
                    {
                        AddToCache(cacheKey, realReturn.ReturnValue);
                    }
                    return realReturn;
                }

                var cachedReturn = input.CreateMethodReturn(cachedResult, input.Arguments);

                return cachedReturn;
            }
        }

        private object getCachedResult(string cacheKey)
        {
            //When the method uses input that is not serializable 
            //we cannot create a cache key and can therefore not 
            //cache the data.
            if (cacheKey == null)
            {
                return null;
            }

            object cachedValue = !storeOnlyForThisRequest ? HttpRuntime.Cache.Get(cacheKey) : HttpContext.Current.Items[cacheKey];
            var cachedValueCast = cachedValue as CacheHandler;
            if (cachedValueCast != null)
            {
                //This is an object that is reloaded when it is being removed.
                //It is therefore wrapped in a CacheHandler-object and we must
                //unwrap it before returning it.
                return cachedValueCast.cachedData;
            }
            return cachedValue;
        }

        private static bool TargetMethodReturnsVoid(IMethodInvocation input)
        {
            var targetMethod = input.MethodBase as MethodInfo;
            return targetMethod != null && targetMethod.ReturnType == typeof (void);
        }

        private void AddToCache(string key, object valueToCache)
        {
            if (key == null)
            {
                //When the method uses input that is not serializable 
                //we cannot create a cache key and can therefore not 
                //cache the data.
                return;
            }

            if (!storeOnlyForThisRequest)
            {
                HttpRuntime.Cache.Insert(
                    key,
                    valueToCache,
                    null,
                    System.Web.Caching.Cache.NoAbsoluteExpiration,
                    expirationTime,
                    CacheItemPriority.Normal, null);
            }
            else
            {
                HttpContext.Current.Items[key] = valueToCache;
            }
        }
    }

    /// <summary>
    /// This interface describes classes that can be used to generate cache key strings
    /// for the <see cref="CacheHandler"/>.
    /// </summary>
    public interface ICacheKeyGenerator
    {
        /// <summary>
        /// Creates a cache key for the given method and set of input arguments.
        /// </summary>
        /// <param name="method">Method being called.</param>
        /// <param name="inputs">Input arguments.</param>
        /// <returns>A (hopefully) unique string to be used as a cache key.</returns>
        string CreateCacheKey(MethodBase method, object[] inputs);
    }

    /// <summary>
    /// The default <see cref="ICacheKeyGenerator"/> used by the <see cref="CacheHandler"/>.
    /// </summary>
    public class DefaultCacheKeyGenerator : ICacheKeyGenerator
    {
        private readonly LosFormatter serializer = new LosFormatter(false, "");

        #region ICacheKeyGenerator Members

        /// <summary>
        /// Create a cache key for the given method and set of input arguments.
        /// </summary>
        /// <param name="method">Method being called.</param>
        /// <param name="inputs">Input arguments.</param>
        /// <returns>A (hopefully) unique string to be used as a cache key.</returns>
        public string CreateCacheKey(MethodBase method, params object[] inputs)
        {
            try
            {
                var sb = new StringBuilder();

                if (method.DeclaringType != null)
                {
                    sb.Append(method.DeclaringType.FullName);
                }
                sb.Append(':');
                sb.Append(method.Name);

                TextWriter writer = new StringWriter(sb);

                if (inputs != null)
                {
                    foreach (var input in inputs)
                    {
                        sb.Append(':');
                        if (input != null)
                        {
                            //Diffrerent instances of DateTime which represents the same value
                            //sometimes serialize differently due to some internal variables which are different.
                            //We therefore serialize it using Ticks instead. instead.
                            var inputDateTime = input as DateTime?;
                            if (inputDateTime.HasValue)
                            {
                                sb.Append(inputDateTime.Value.Ticks);
                            }
                            else
                            {
                                //Serialize the input and write it to the key StringBuilder.
                                serializer.Serialize(writer, input);
                            }
                        }
                    }
                }

                return sb.ToString();
            }
            catch
            {
                //Something went wrong when generating the key (probably an input-value was not serializble.
                //Return a null key.
                return null;
            }
        }

        #endregion
    }
}

Microsoft deserves most credit for this code. We've only added stuff like caching at request level instead of across requests (more useful than you might think) and fixed some bugs (e.g. equal DateTime-objects serializing to different values).

Answer 6

Ed, I assume those lists and dictionaries contain almost static data with low chances of expiration. Then there's data that gets frequent hits but also changes more frequently, so you're caching it using the HttpRuntime cache.

Now, you should think of all that data and all of the dependencies between diferent types. If you logically find that the HttpRuntime cached data depends somehow on your GlobalData items, you should move that into the cache and set up the appropriate dependencies in there so you'll benefit of the "cascading expiration".

Even if you do use your custom caching mechanism, you'd still have to provide all the synchronization, so you won't save on that by avoiding the other.

If you need (preordered) lists of items with a really low frequency change, you can still do that by using the HttpRuntime cache. So you could just cache a dictionary and either use it to list your items or to index and access by your custom key.