Is this page caching system reasonable?

Question

I am working on a webbased system where users will click through a lot of pages. Because they will often return to specific pages (for example, an overview of books), I would like to implement some caching mechanism to speed things up.

Before I show the contents of the user, I would store an MD5 checksum of the content in a database, together with the user's ID and the URL of the page.

When he logs in next time, I'll check if there is an entry with the URL and the UserID. If there is, I compare the stored MD5 with the one I'll calculate freshly, and if the MD5s are equal, I send a "not changed" header so the browser does not have do download the whole page.

Answer 1

Have you first profiled the web application to identify where your performance bottlenecks are located?

Beware of unnecessary optimization: if the pages are already loading just fine but you're trying to squeeze out a few percentage points of speed, I personally wouldn't bother with any specialized optimizations.

Using the proposed method, keep in mind that you will still be making database hits, in addition to MD5 computation. First ensure that you have the proper indexes in place in the database to speed up your queries -- this might be all you need to do.

Is the page image-heavy? Are the images stored as BLOBs in the database or on the filesystem? If the images are stored in the database, move them to the filesystem and store the filenames in the database instead. Use a web server dedicated to static content to serve up the images. The static content server can also be a proxy for your app.

If the goal is to reduce bandwidth usage in terms of plaintext content (i.e., HTML, CSS and JavaScript), I suggest simply turning on gzip compression. Also, consider the static content web server for this case.

But to answer your question about whether your proposed solution is reasonable: I imagine it could work, but there are many factors as to whether it is effective on any given site. So one thing you can do is implement the caching system and then run benchmarking tests against your site (e.g., Apache ab) and see if it in fact makes a difference.