Hybrid Rails Caching Options, am I reinventing something?

Question

Well, not really reinventing, however, we have a large content-based website which handles load (after we fixed the SQL pooling issue) to a certain point, then we just run out of steam. A lot of this is due to bad code we are trying to fix up, but a lot is just due to the level of requests etc.

We were considering page caching, because well, it's damn quick (yep... :D) but that doesn't work because we have certain fragments within the page which are specific to the logged in user. But, not all hope is lost...

I was wondering if it would be ideal to do the following:

1. Page level caching, with sweepers to clean out the pages when content is updated.
2. Replace the user-specific fragments with placeholders (and perhaps generic content like... 'View your Account, or Sign Up Here')
3. When the users page loads fire off an async request (AJAX, or, some would call it AJAH), which requests the 'dynamic' fragment, then place the content placeholders with this fragment..

The main issue I can see with this is that users with JS turned off wouldn't see the content, but I honestly don't think we would be too affected by this, and IMHO people who disable javascript, for the most part, are idiots (yes, I said it!).

I'd also be interested to know if I'm (no doubt) reinventing something, and if anyone can point me to a site which is already doing something like this it would be appreciated.

Thanks awesome SO community!

Answer 1

Ryan Bates covered this technique in Railscast 169: Dynamic Page Caching. It's worth taking a look at.

Answer 2

Have you thought about server-side fragment caching? I've used it extensively, and it couldn't be more awesome. You could simply cache the content 'fragments' and render normally whatever depends on the logged in user.

There are many good resources for fragment caching, I'd start at the documentation: http://api.rubyonrails.org/classes/ActionController/Caching/Fragments.html

Also very good from the Scaling Rails series: http://railslab.newrelic.com/2009/02/09/episode-7-fragment-caching

Answer 3

When serving static content or cached content starts to slow down serving the real working processes put a reverse proxy as frontend to your application. It will free processes to do real work and reduce slowdowns due to file system caches becoming inefficient. And it will help you to make "client side caching" shareable to multiple visitors. Have a look at the fantastic screen cast series about caching and scaling by NewRelic: http://railslab.newrelic.com/scaling-rails