Custom webserver caching

Question

I'm working with a custom webserver on an embedded system and having some problems correctly setting my HTTP Headers for caching.

Our webserver is generating all dynamic content as XML and we're using semi-static XSL files to display it with some dynamic JSON requests thrown in for good measure along with semi-static images. I say "semi-static" because the problems occur when we need to do a firmware update which might change the XSL and image files.

Here's what needs to be done: cache the XSL and image files and do not cache the XML and JSON responses. I have full control over the HTTP response and am currently:

1. Using ETags with the XSL and image files, using the modified time and size to generate the ETag
2. Setting Cache-Control: no-cache on the XML and JSON responses

As I said, everything works dandy until a firmware update when the XSL and image files are sometimes cached. I've seen it work fine with the latest versions of Firefox and Safari but have had some problems with IE.

I know one solution to this problem would be simply rename the XSL and image files after each version (eg. logo-v1.1.png, logo-v1.2.png) and set the Expires header to a date in the future but this would be difficult with the XSL files and I'd like to avoid this.

Note: There is a clock on the unit but requires the user to set it and might not be 100% reliable which is what might be causing my caching issues when using ETags.

What's the best practice that I should employ? I'd like to avoid as many webserver requests as possible but invalidating old XSL and image files after a software update is the #1 priority.

Answer 1

Are we working on the same project? I went down a lot of dead ends figuring out the best way to handle this.

I set my .html and my .shtml files (dynamic JSON data) to expire immediately. ("Cache-Control: no-cache\r\nExpires: -1\r\n") Everything else is set to expire in 10 years. ("Cache-Control: max-age=290304000\r\n")

My makefile runs a perl script over all the .html files and identifies what you call "semi-static" content (images, javascript, css.) The script then runs a md5 checksum on those files and appends the checksum to the file:

<script type="text/Javascript" src="js/all.js?7f26be24ed2d05e7d0b844351e3a49b1">

Everything after the question mark is ignored, but no browser will cache it unless everything between the quotes matches.

I use all.js and all.css because everything's combined and minified using the same script.

Out of curiosity, what embedded webserver are you using?

Answer 2

Try Cache-Control: no-store. no-cache tells the client that the response can be cached; it just generally isn't reused unless the cache can't contact the origin server.

BTW, setting an ETag alone won't make the response cacheable; you should also set Cache-Control: max-age=nnn.

You can check how your responses will be treated with http://redbot.org/