scaling up using memcached to store gzipped file

Question

I have a web service that serves a javascript source file to browsers. The javascript file is compiled by php in real time with some file includes. The resulting file changes only a few times a day. It is gzipped by apache before being served to browsers.

I'd like to use memcached to serve it from memory to avoid disk and cpu loads of file includes and repeating gzip.

My question is how can I gzip and store a javascript file in memcached and serve it (from php)?

It looks to me that if the file is not in store I should get the file by include, fetch the output buffer, gzip it and store it in memcached. Am I correct? Are there some headers I should add to the output or anything else I should do to make the result compatible with the gzipped file currently served by my apache?

Update: I didn't mention that the file has several variations controlled by a url parameter which cause different def files to be included. Each of the variations is rather static as described above.

Answer 1

It's best if you use the appropriate tools for the tasks they were designed.

In your case, you have a slowly-changing file that would be served statically. By that i mean that it doesn't depend on the query, but it's the same for every query during its validity time.

For that, use a good caching webserver or put a cache/load balancer in front of your server(s). Either Squid, Varnish or NginX would do. In fact, either of these can serve a static file just as fast as memcached. Just be sure of gzipping the file yourself and serving it like that.

memcached beats any of these in writing speed, not serving. so it's the best answer for content that changes with the query parameters, so it has to be generated and expired constantly (and overlapping).

Since you'll zip and write the file only a few times, the zip/write time is irrelevant. After that, the many levels of cache will get you the best performance.

Also note that since mencached isn't a webserver, it's cache isn't as 'close' to the client as a proper web cache would, negating any hypothetical read performance advantage

Answer 2

Perhaps the right solution is to use Apache's own built-in caching on your web servers themselves.

After all, it's not like there are lots of different variations of this, or that it changes frequently. It wouldn't be (I guess) a big hit if each server in your farm generates its own copy from time to time.

Apache's caching respects the "Expires" header etc, in the output, so it's easy enough to simply turn it on for those URIs and output the appropriate headers.

The "advantage" of memcached is that your memcached servers form a pool which looks like a single cache and can be shared by all of your web nodes; this is not really relevant if you have only a very small number of items (in this case it sounds like you have one or just a few).