Working around memcached's lack of wildcard expiration in Rails fragment caching

Question

I'm working on adding fragment caching to a Rails 3 site that has both logged in and anonymous users, and need to control when parts of the page expire based on when content displayed throughout the site is updated. Midway into this, I discovered that memcached doesn't support regexps for expire_fragment, so now I'm looking for ways around this.

For anonymous users, this is not a huge problem. I create cache keys like 'frontpage-new-shows' or 'schedule-weekly-forecast-10/24/10' and expire them when a new entry in the Show model is made, or when a show that airs within the next week is modified via sweepers. That works just fine.

My problem is with users who are logged in. They each have their own customized schedule, so my keys look like 'schedule-user-jschuur-10/10/24' with jschuur being the username to the account. I understand that with the date based convention, they expire naturally, but I also want to explicitly expire all fragments for cached schedules during the day when show related changes occur for something in your schedule for that day(or week).

As it turns out, memcached can't do the kind of wildcard expiration that I need here (schedule-user-.*-10/10/24). This means I need some kind of solution that stores all the keys issued in a central lookup key in memcached, or have Rails/Ruby somehow do this internally based on the keys I've sent to memcached for me.

I'm assuming the former is the better solution. My question is, how do I do this in the most efficient way, so I don't blow all the time saved not hitting the DB any more. I could just store an array or hash in memcached, retrieve the whole thing, loop over if for matches, remove them and store the hash back. That sounds like it might work great for a few hundred or even a thousand user, but is this really the best way to go?

Has someone already tackled this problem and released a solution?

One thing to consider is that almost all of my caching is currently done with statements in the view, geared towards ActiveRelations prepared queries that haven't fired yet, like this one:

<% if current_user %>
  <% if current_user.saved_shows_count %>
    <% cache "schedule-user-#{current_user.username}-#{(Time.now + 3.hours).to_date.end_of_week.strftime('%D')}" do %>
      <% if @shows.any? %>
        <%= render :partial => "schedule/schedule", :locals => { :shows => @shows } %>
      <% end %>
    <% end %>
...

The site is hosted on Heroku and I'm using the dalli gem as a client.

Answer 1

EDIT: There are some issues with this as written with the dalli gem, doing a read raises "marshal data too short" or "invalid marshalled data in memcache" for me. Not sure if it will for you as well.

Although a little hackish, you could use a namespace and then increase the namespace, thus invalidating all keys created using it. To do this, first set the namespace in an initializer (or wherever) with Rails.cache.write("frontpage-new-shows-namespace", "1", {:raw => true}). When you set each cache, add Rails.cache.read("frontpage-new-shows-namespace") into the hash key wherever you would like (it doesn't really matter where). Expiring all cache keys created in a namespace is as easy as Rails.cache.increment("frontpage-new-shows-namespace", 1). You might be able to add Rails.cache.read("frontpage-new-shows-namespace") as a global variable, eliminating memcache searches for creating cache keys, and then update this variable whenever you do a cache namespace increment.

Good luck, and I hope this helps.