Page-view-counter with Rails and Memcached

views:

665

answers:

Page-view-counter with Rails and Memcached

Hi!

I'm trying to implement a page-views-counter with Rails and memcached. Every time I render a page, through rails I increase a memcached key (key.incr is atomic). My main worry is the possibility where this key gets expired or deleted from memcached before I update my DB record. Even if I visit all the keys with frequency greater than their expiration time, memcached might delete a key in the meantime because of full memory.

Any suggestions? Thank you Dimitris

+1 A:

If you want that data to be persitent, you must not write it to memcache (which is a caching mecanism, and not a data persistance storage).

Basically, what I'd probably do would be like this :

When trying to get a counter for a page :
- Check if it's stored in memcache
  - if yes, use it
  - if not, fetch it from the DB and store it to memcache
When trying to write a counter (i.e. counter += 1) :
- Write the data to the database (update ... set counter = counter + 1 where... )
- select the data back from the database ; wrapping both update and select in a transaction might help : isolation is something databases do well.
  - and immediatly write it to memcache, so it's up to date for the next "get" operation

But I would not use memcache for persistance :

I would never write to memcache any data that has not been written to the database
persistance is the job of the database ; not of a caching engine.

Pascal MARTIN 2010-02-26 17:14:46

Actually I want to avoid the overhead of multiple update of the counter. I'm not worrying for the selects..

Dimitris 2010-02-27 00:19:54

Many databases buffer multiple updates to the same field. InnoDB certainly does this. Each update is written serially to the transaction log, which is super-fast on most hard disks, but the actual updates are written out much less frequently.

Seun Osewa 2010-04-25 14:58:49

Turning every page read into a DB write is terrible for anything other toy sites and relying on DB logging to handle the load is playing with fire. It can, e.g., blow out your system disk cache which is how most DBs maintain acceptable performance these days. Or, it can grind to a halt when a larger txn commits and has to fsync a table.

Logan Bowers 2010-10-19 22:20:36

+1 A:

I would go with redis as a memcached replacement. It's perfect for realtime stats. It gives you the speed and atomic increments that you want, plus it persists. Problem solved.

Ben Marini 2010-02-26 17:15:36

Do you people have used Redis? It's the first time hearing about it! Not that much documentation nor citations around! Do you suggest switching from memcached to redis? or working in parallel is not that bad?

Dimitris 2010-02-26 21:38:22

I use Redis at my work as it powers Resque (http://github.com/blog/542-introducing-resque), a background job processing library. As for running in parallel, sure. That'd be an easy way of testing it out without worrying about changing your existing cache setup. You'll want the redis gem (http://rubygems.org/gems/redis) to interface with your redis server.

Ben Marini 2010-02-26 22:21:15

So the best way you suggest for avoiding the overhead of the updates is with redis? If you wanted to implement a page views counter that would have been your way?

Dimitris 2010-02-27 00:21:39

I would give redis a good hard look. People are already using it for just this sort of thing. Check out Vanity (http://vanity.labnotes.org/). It's a rails add on that leverages redis to do realtime tracking. In fact, you could probably implement your page view counter in just a few lines of code using vanity. It even comes with a nice little dashboard.

Ben Marini 2010-02-27 01:02:00

As for redis's performance, take a look at the impressive benchmarks: http://code.google.com/p/redis/wiki/Benchmarks

Ben Marini 2010-02-27 01:03:15

ansaurus

tags:

views:

answers:

Page-view-counter with Rails and Memcached

related questions