Serializing/Deserializing generics dictionaries where the key is reference type with JSON.NET

Question

I am trying to serialize something like this Dictionary. I also want to preserve the type information. By default, it appears json.net calls ToString() on the key. So that the resulting json is something like

{$type':'System.Collections.Generic.Dictionary`2[[Person, myAssembly],[System.Int32, mscorlib]], 
    'Person',100, 
    'Person',200
}

I have attempted to create a custom convert that detects the dictionary through a customer contract resolver:

public class CustomContractResolver : DefaultContractResolver
{
 protected override JsonConverter ResolveContractConverter(Type objectType)
 {

  if (typeof(IDictionary).IsAssignableFrom(objectType) && objectType.IsGenericType)
  {
   var keyType = objectType.GetGenericArguments()[0];
   if (!keyType.IsValueType && keyType != typeof(string))
   {
    return new DictionaryWithClassKeyConverter();;
   }
  }

  return base.ResolveContractConverter(objectType);
 }
}

With this I can control the serialization, but it's skipped over when I attempt to deserialize. The type that I am emitting when serializing is identical to the default type value.

Any suggestions?