JS/CSS caching problem

Question

Hello,

To reduce http requests, as well as make the rollout process easier for js/css files, I've setup my php app to do concatenation of js and css files. All included files are sequentially combined into one large file using 2 letter codes for each file (ex. home.js code = "AA"). To generate the names I combine all the codes in the order they are loaded, add an underscore, then the timestamp (ex "AABACE_12345678.js"). Then in the directory they are served out of i'm using an apache directive in htaccess to parse the files with php (sing sethandler) so I can control the headers, and return 301 headers on not changed.

Unfortunately this is not working... I have tested the crap out of it with firebug's Net panel and liveHTTPHeaders. The page is loading the file every time, even if it should have it in cache. Here's an example of the response headers one of the files returns:

Date                Wed, 14 Jul 2010 17:00:28 GMT
Cache-Control       private
Content-Encoding    gzip
Etag                "4b4a6d50f9acf924b9dce14e415f5c78"
Expires             Wed, 28 Jul 2010 17:00:28 GMT
Vary                Accept-Encoding
X-Powered-By        PHP/5.2.6
Last-Modified       Wed, 14 Jul 2010 16:59:35 GMT
Content-Length      39305
Content-Type        text/css

Note: i've omitted some non-caching related headers, like keepAlive, server, etc

I should also note that I'm conditionally gzipping the content, hence the private cache-control header. What am I missing here? The content should be cached because of the expires header. I'm generating the etag from an md5 has of filename+timestamp, so that never changes for the file. Last-Modified never changes, Expires never changes....

Any help/suggestions would be GREATLY appreciated!

[EDIT]

So after further testing, my css files ARE getting cached, but my javascript is not. If I have two files, with IDENTICAL caching headers (etag,expires,last-modified,cache-control) but the only thing different is the content type (one "text/css" one "text/javascript"), ONLY the css is getting cached... frustrating

Answer 1

It's an interesting exercise, but it seems to me that in the long run if your site started getting hammered that the server-side combination and renaming of files could add its own overhead that might be worse than just loading a few extra files. I'd be interested to see some stats to know what effects it has on server load.

I did some redesign work for an international bank--my team redesigned over 1,000 major credit card websites, some of which got more than 1 million uniques a day (fairly serious traffic) There was one person assigned entirely to standards and optimization. In the end, we did the basics like minifying js files and the like, and made extensive use of sprites for the reasons you're trying to overcome. But upon analyzing the ramifications of going to levels you speak of, it was determined that the cost in man hours would significantly outdo the benefit in saved bandwidth costs--even in a situation where a 5k savings/file would equal 5gb of bandwidth daily.

Do you have a copy of the "old way" of doing things? An analysis comparing old vs new once you get this licked would be cool.