Best way to generate static version of "dynamic" web site

Question

I have a website that is dynamic in the sense that a lot of data is generated from a database, but the contents of the database changes rarely (about 1-3 times a week). These changes are manual and controlled.

Instead of having the overhead of a dynamic website, I prefer to use a static pages. I'm debating what is the best solution:

curl/wget/spider

This question mentions it. The disadvantages I see might be:

• manual clean up needed (links, missing images, etc.)
• cannot mix static and dynamic pages

proxy

I could use a proxy to cache the static pages for a certain number of days. Disadvantages:

• hard to manage the cache of each page
• need to clear the cache after each manual change?

Use program to generate static pages

My current choice: I use perl programs to generate static pages from dynamic content. This doesn't scale very well as I have to hard code a lot of HTML, especially the page structure

Any other ways to do it? What would you/do you prefer?

Answer 1

Any particular reason you want to do it this way instead of just setting up a database caching solution to stop the queries from actually having to hit the database?

Whether it's possible or not depends on the amount of dynamic data that's on your site, and the amount of memory available in your server, but it wouldn't have any of the problems you're worried about.

Answer 2

Memcache base full-page cache with long expire time. Tag extension could allow you to invalidate only selected range of pages.

Answer 3

I would do it the same way you're doing it right now, using a script to generate static pages. You can use a templating system to avoid having to write new HTML every time.

Answer 4

You have not mentioned how important it is to show the changed data as soon as possible to your user.

We have used proxy cache successfully for our website to handle dynamic pages which gets lots of hits. Depending upon how soon we want the updated data to be seen by customer we kept different cache age for each categories.