Rails with Memcache returning wrong cached object?

Question

I have a fairly large Rails app, which uses memcached on a seperate server as its cache store.

The problem is that I randomly get errors in the production environment which seem to indicate that memcached is returning an incorrect object.

Examples:

In this example, current_site is a helper method which accesses a method on the Site model that uses Rails.cache to cache the model

ActionView::TemplateError in ListingsController#edit
undefined method `settings' for #<String:0xb565f8a0>

On line #12 of app/views/layouts/site.html.erb

    9:         <meta name="robots" content="noodp, all" />
    10:         <meta name="distribution" content="Global" />
    11: 
    12:         <% unless current_site.settings[:google_webmaster_verification_code].blank? %>
    13:         <meta name="verify-v1" content="<%= current_site.settings[:google_webmaster_verification_code] %>" />
    14:         <% end %>
    15:

contrasted with....

ActionView::TemplateError in ApplicationController#not_found
undefined method `settings' for #<Category:0xd5c6c34>

On line #12 of app/views/layouts/site.html.erb

    9:         <meta name="robots" content="noodp, all" />
    10:         <meta name="distribution" content="Global" />
    11: 
    12:         <% unless current_site.settings[:google_webmaster_verification_code].blank? %>
    13:         <meta name="verify-v1" content="<%= current_site.settings[:google_webmaster_verification_code] %>" />
    14:         <% end %>
    15:

When both should be returning a Site model!

Another example of cache behaving strangely:

ActionView::TemplateError in AccountsController#show
can't convert Category into String

On line #141 of app/views/layouts/site.html.erb

    138:        <li<%=  class="first" if i == 0 %>><%= link_to top_level_category.title, top_level_category.path %></li><% end %>
    139:       </ul>
    140:      <% end %>
    141:      <% cache bottom_pages do %>
    142:       <ul><% Page.top_level.active.show_in_navigation.find(:all, :include => :slugs).each_with_index do |top_level_page, i| %>
    143:        <li<%=  class="first" if i == 0 %>><%= link_to top_level_page.title, top_level_page.path %></li><% end %>
    144:       </ul>

Has anyone encountered something like this before? Anyone have thoughts on diagnosing this unreplicable problem!? I've tried switching out memcached client gems, thinking maybe it was a weird bug, but this didn't have any effect! Thanks.

Answer 1

A few things that might help:

• Add instrumentation/logging to current_site to see exactly what is being returned.
• How are you specifying keys in memcache? You could accidentally be using the same key in two different places for two different objects.
• Use memcached-tool host:port dump > /tmp/keys to look at what's actually in your memcache.
• Your memcached is behind a firewall and not exposed on a public IP, right?

Answer 2

This was being caused by Passenger sharing its connection to the Memcached server. Check http://www.modrails.com/documentation/Users%20guide.html#_example_1_memcached_connection_sharing_harmful.

The fix was simply to change Passenger's Rails spawn to conservative.

Answer 3

Yes, I've had this happen. With me, it was because I was doing Rails.cache.fetch(key) and key was blank.

I've done some playing around in the Rails console and with the following:

Rails.cache.read validkey # Get back the proper data

Rails.cache.fetch('') { 'abc' } # Error in rails log: 'MemCacheError ():'

Rails.cache.read validkey # Get back nil

Rails.cache.read validkey # May get back proper data

Answer 4

We have a similar problem, unfortunately changing Passenger's Rails spawn to conservative does not help :( It's driving me crazy, it's totally random! Ideas?