AppFabric: Could not contact the cache service

Question

I'm trying to use AppFabric with NHibernate as my second level cache provider but I'm getting the following error: ErrorCode:Initialization: Could not contact the cache service. Contact administrator and refer to product help documentation for possible reasons.

I presume that the problem is with my configuration in web.config:

    <section name="dcacheClient" 
             type="Microsoft.ApplicationServer.Caching.DataCacheClientSection, Microsoft.ApplicationServer.Caching.Core"
             allowLocation="true" 
             allowDefinition="Everywhere"/>
...
  <dcacheClient deployment="routing" localCache="False">
    <localCache isEnabled="false" sync="TimeoutBased" ttlValue="300" />
    <hosts>
      <host name="localhost" cachePort="22233" cacheHostName="AppFabricCachingService" />
    </hosts>
  </dcacheClient>

I've downloaded the NHibernate.Caches source code to try and discover where the problem lies and the exception is being thrown in the VelocityClient constructor when the GetCache method is called:

  public VelocityClient(string regionName, IDictionary<string, string> properties)
  {
      region = regionName.GetHashCode().ToString(); //because the region name length is limited
      var cacheCluster = new CacheFactory();
      cache = cacheCluster.GetCache(CacheName);
      try
      {
          cache.CreateRegion(region, true);
      }
      catch (CacheException) {}
  }

If I add a watch to the cacheCluster variable, I can find a _servers private variable which has one System.Data.Caching.EndpointID which has the MyURI property set to net.tcp://localhost:22234/AppFabricCachingServive which I presume has come from the configuration in web.config.

If you don't know the exact cause of the problem but have some ideas on how to go about troubleshooting this problem, that would be much appreciated as well.

Additional Info

---

I get the following results from the command, Get-CacheHostConfig -HostName tn-staylor-02 -CachePort 22233:

HostName        : tn-staylor-02
ClusterPort     : 22234
CachePort       : 22233
ArbitrationPort : 22235
ReplicationPort : 22236
Size            : 3001 MB
ServiceName     : AppFabricCachingService
HighWatermark   : 90%
LowWatermark    : 70%
IsLeadHost      : True

So I think the values I've got configured in web.config are OK.

---

Googling this problem and investigating how to set up AppFabric in the first place, I have come across two slightly different ways of how to configure the cache in web.config. The way I have described above and the way Hanselman has it in his AppFabric blog post

I actually started with it like this however, I got the following error which is how I came to have it configured how I have it now:

ErrorCode:"dcacheClient" tag not specified in the application configuration file. Specify valid tag in configuration file.

---

Full stack trace of the exception that gets thrown in VelocityClient:

System.Data.Caching.CacheException occurred Message="ErrorCode:\"dcacheClient\" tag not specified in the application configuration file. Specify valid tag in configuration file." Source="CacheBaseLibrary" ErrorCode="ERRCMC0004" StackTrace: at System.Data.Caching.ClientConfigFile.ThrowException(String errorCode, String param) at System.Data.Caching.ClientConfigReader.GetDeployementMode() at System.Data.Caching.ClientConfigurationManager.InitializeDepMode(ClientConfigReader cfr) at System.Data.Caching.ClientConfigurationManager.Initialize(String path) at System.Data.Caching.ClientConfigurationManager..ctor() at System.Data.Caching.CacheFactory.InitCacheFactory() at System.Data.Caching.CacheFactory.GetCache(String cacheName) at NHibernate.Caches.Velocity.VelocityClient..ctor(String regionName, IDictionary`2 properties) in C:\Source\Projects\NHibernate.contrib\trunk\src\NHibernate.Caches\Velocity\NHibernate.Caches.Velocity\VelocityClient.cs:line 67 InnerException:

---

EDIT: Added output from get-cachehost as requested by @PhilPursglove

Output from get-cachehost:

HostName : CachePort      Service Name            Service Status Version Info
--------------------      ------------            -------------- ------------
tn-staylor-02:22233       AppFabricCachingService UP             1 [1,1][1,1]

---

SOLUTION: @PhilPursglove was spot on. The NHibernate velocity provider was using old dll's so upgrading them and making a few code changes resolved my problems. I thought I would include my complete solution here.

1. Downloaded the NHibernate.contrib source from the SVN repository at https://nhcontrib.svn.sourceforge.net/svnroot/nhcontrib/trunk
2. Opened up the NHibernate.Caches.Everything solution and removed the references to the old velocity dll's from the NHibernate.Caches.Velocity project.
3. Added references to the App Fabric dll's which were installed when I installed App Fabric. This isn't the normal case of adding a reference to an assembly in the GAC, but this article describes how to do it.
4. Adding the new references meant that the VelocityClient class no longer compiled. With a little bit of help from this I came up with the version of VelocityClient.cs below.
5. I added a reference to the new version of NHibernate.Caches.Velocity to my project, made the changes below to my configuration and everything worked.

VelocityClient.cs

using System;
using System.Collections.Generic;
using Microsoft.ApplicationServer.Caching;
using log4net;
using NHibernate.Cache;
using CacheException = Microsoft.ApplicationServer.Caching.DataCacheException;
using CacheFactory = Microsoft.ApplicationServer.Caching.DataCacheFactory;

namespace NHibernate.Caches.Velocity
{
    public class VelocityClient : ICache
    {
        private const string CacheName = "nhibernate";
        private static readonly ILog log;
        private readonly DataCache cache;
        private readonly string region;
        private Dictionary<string, DataCacheLockHandle> locks = new Dictionary<string, DataCacheLockHandle>();

        static VelocityClient()
        {
            log = LogManager.GetLogger(typeof (VelocityClient));
        }

        public VelocityClient() : this("nhibernate", null) {}

        public VelocityClient(string regionName) : this(regionName, null) {}

        public VelocityClient(string regionName, IDictionary<string, string> properties)
        {
            region = regionName.GetHashCode().ToString(); //because the region name length is limited
            var cacheCluster = new CacheFactory();
            cache = cacheCluster.GetCache(CacheName);
            try
            {
                cache.CreateRegion(region);
            }
            catch (CacheException) {}
        }

        #region ICache Members

        public object Get(object key)
        {
            if (key == null)
            {
                return null;
            }
            if (log.IsDebugEnabled)
            {
                log.DebugFormat("fetching object {0} from the cache", key);
            }

            DataCacheItemVersion version = null;
            return cache.Get(key.ToString(), out version, region);
        }

        public void Put(object key, object value)
        {
            if (key == null)
            {
                throw new ArgumentNullException("key", "null key not allowed");
            }
            if (value == null)
            {
                throw new ArgumentNullException("value", "null value not allowed");
            }

            if (log.IsDebugEnabled)
            {
                log.DebugFormat("setting value for item {0}", key);
            }

            cache.Put(key.ToString(), value, region);
        }

        public void Remove(object key)
        {
            if (key == null)
            {
                throw new ArgumentNullException("key");
            }
            if (log.IsDebugEnabled)
            {
                log.DebugFormat("removing item {0}", key);
            }

            if (Get(key.ToString()) != null)
            {
                cache.Remove(region, key.ToString());
            }
        }

        public void Clear()
        {
            cache.ClearRegion(region);
        }

        public void Destroy()
        {
            Clear();
        }

        public void Lock(object key)
        {
            DataCacheLockHandle lockHandle = null;

            if (Get(key.ToString()) != null)
            {
                try
                {
                    cache.GetAndLock(key.ToString(), TimeSpan.FromMilliseconds(Timeout), out lockHandle, region);
                    locks.Add(key.ToString(), lockHandle);
                }
                catch (CacheException) {}
            }
        }

        public void Unlock(object key)
        {
            DataCacheLockHandle lockHandle = null;

            if (Get(key.ToString()) != null)
            {
                try
                {
                    if (locks.ContainsKey(key.ToString()))
                    {
                        cache.Unlock(key.ToString(), locks[key.ToString()], region);
                        locks.Remove(key.ToString());
                    }
                }
                catch (CacheException) {}
            }
        }

        public long NextTimestamp()
        {
            return Timestamper.Next();
        }

        public int Timeout
        {
            get { return Timestamper.OneMs * 60000; } // 60 seconds
        }

        public string RegionName
        {
            get { return region; }
        }

        #endregion
    }
}

NHibernate.config:

...
    <property name="cache.provider_class">NHibernate.Caches.Velocity.VelocityProvider, NHibernate.Caches.Velocity</property>
    <property name="cache.use_second_level_cache">true</property>
    <property name="cache.use_query_cache">true</property>
...

web.config

...
    <section name="dataCacheClient"
             type="Microsoft.ApplicationServer.Caching.DataCacheClientSection, Microsoft.ApplicationServer.Caching.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"
             allowLocation="true"
             allowDefinition="Everywhere"/>
...
  <dataCacheClient>
    <!-- cache host(s) -->
    <hosts>
      <host
         name="localhost"
         cachePort="22233"/>
    </hosts>
  </dataCacheClient>
...

I didn't make any further changes to my App Fabric configuration or anything.

Answer 1

I think there are two possible culprits here:

1. In your web.config under the hosts element, you're listing localhost - I'd try swapping that out for the actual server name tn-staylor-02

2. That exception stack trace refers to CacheBaseLibrary - I don't know a great deal (read: anything!) about NHibernate but I would hazard a guess that that cache might not be built with the release version of AppFabric - CacheBaseLibrary was an assembly that appeared in the CTPs and betas but I didn't think it was used in the RTM version. Note that in the section element for dcacheclient, it refers to the Microsoft.ApplicationServer.Caching.Core assembly.