etags and server farm

Question

I gathered from the much famed scaling rails screencasts that at some point when your site gets big and bigger, proxy caching is the way to go. Proxy caching uses etag, among other things and since etags can be more specific and strong validator is perhaps the way to go. However, I also hear that in server farm scenarios the etag is not the right solution because it can vary across servers (How?)

This seems contradictory i.e. most likely one is implementing e-tag based proxy caching if they are running a large load balanced server farms. So if e-tag fails in this situation how do they do it? :last_modified isn't really a great option.

In a rails app let's say if my etags in a post index action is

:etag => "all_posts_#{Post.count}".

will this vary from server to server if it's a load balanced server farm?

Answer 1

Usually when they talk about Etags varying across servers it's in relation to static connect served up by Apache. By default Apache includes the file's inode in the Etag. If the files are not on a shared resource (like a NFS exported NAS), then the file's inode would be different on each server. Typically, the recommendation is to configure Apache like:

FileETag MTime Size

but even that has the possibility of differences if the modification time varies across the servers.

However, for non-static content, you are generating the Etag in your code, so it would be the same across multiple servers.