Getting ETags right

Question

I've been reading this book (highly recommended):

And I have a particular question about the ETag chapter. The author says that ETags might harm performance and that you must tune them finely or disable them completely.

I understand the risks, but is it that hard to get ETags right?

I've just made an application that sends an ETag whose value is the MD5 hash of the response body. This is a simple solution, easy to achieve in many languages.

My question is: Is this wrong? if so, why? and why the author (who obviously outsmarts me by many orders of magnitude) does not propose such a simple solution?

Thanks

EDIT Please Read!

People are misunderstanding the question. I already know what ETags are, I'm just asking if a MD5 hash of the response body is fine enough, and if so why is not encouraged on the book? This last question is hard to answer unless you are the author :) so I'm trying to find the weak points of using a MD5 hash as an ETag.

Answer 1

I think the perceived problem with ETAGS is probably that your browser has to issue and parse a (simple and small) request / response for every resource on your page to check if the etag value has changed server side.

I personally find these extra small roundtrips to the server acceptable for often changing images, css, javascript (the server does not need to resend the content if the browser's etag is current) since the mechanism makes it quite easy to mark 'updated' content.

Answer 2

Having not read the book, I can't speak on the author's precise concerns.

However, the generation of ETags should be such that an ETag is only generated once when a page has changed. Generating an MD5 hash of a web page costs processing power and time on the server; if you have many clients connecting, it could start to cause performance problems.

Thus, you need a good technique for generating ETags only when necessary and caching them on the server until the related page changes.

Answer 3

ETag is similar to the Last-Modified header. It's a mechanism to determine change by the client.

Arguably, an ETag that JUST HAPPENS to be the Last Modified date (i.e. the same text) meets all the criteria necessary for an ETag. It simply needs to be a unique value representing the state of a resource. Not unique across the entire domain of resources, simply within the resource.

Now, technically, an ETag has "infinite" resolution compared to a Last-Modified header. Last-Modified only changes at a granularity of 1 second, whereas an ETag can be sub second.

You can implement both ETag and Last-Modified, or simply one or the other (or none, of course). If you Last-Modified is not sufficient, then consider an ETag.

Mind, I would not set ETag for "every" resource. Basically, I wouldn't set it for anything that has no expectation of being cached (dynamic content notably). There's no point in that case, just wasted work.

Edit: I see your edit, and clarify.

MD5 is fine. The only downside is calculating MD5 all the time. Running MD5 on, say, a 200K PDF file, is expensive. Running MD5 on a resource that has no expectation of being cashed is simply wasteful (i.e. dynamic content).

The trick is simply that whatever mechanism you use, it should be as cheap as Last-Modified typically is. Last-Modified is, again, typically, a property of the resource, and usually very cheap to access.

ETags should be similarly cheap. If you are using MD5, and you can cache/store the association between the resource and the MD5 hash, then that's a fine solution. However, recalculating the MD5 each time the ETag is necessary, is basically counter to the idea of using ETags to improve overall server performance.