Sitemap for dynamic content?

Question

I am developing a webapp using codeigniter (mvc) and php. I was wondering how do I create a sitemap for search engines when my contact is dynamic? Are there best practices for this, or is there a way to automate the process? I ask because content will be constantly and rapidly added so I'd like the sitemap be recent. Thanks

Update: to help answer my question, the type of site that I am creating is a public blog type site where users post articles.

Update2: Just to elaborate on my confusion: I may not understand exactly how a sitemap is structured. On a site like stackoverflow for example, as users add new questions (and thus, new pages get created) does the sitemap change and need to be resubmitted? How important is it for a site like this one to even have a sitemap?

Answer 1

Depending on what type of site you creating you can go about it in several ways.

For example if you were making a news site. You could have a single page that links to each news article's permalink. If you could detail what kind of site/ its content is we could help better.

Answer 2

I have a similar situation to yours, except I develop in Zend Framework and Doctrine instead of Code Igniter.

The way we solved this issue, was to build a Sitemap model that acts as a Doctrine Nested Set. It's basically just a few fields, like an id, parent_id, title, uri, created, updated. You can add more fields if you need, but that's the basic premise.

We then using Zend_Navigation to render this tree into a menu for the site.

Zend_Navigation also has a helper method for automatically generating an XML sitemap, so we have a controller and view that is setup to render that XML document.

Then we have written a Zend Action Controller Plugin, that monitors the sitemap controller, and when updates are made, it checks against the last update, and pings Google that the sitemap was updated.

All of these concepts can easily be ported into CI, I would imagine.

If there isn't a sitemap view helper in CI, you can probably use the Zend Navigation classes, since Zend is loosely decoupled.

Or you can just generate the xml document with CI, using the standard for sitemaps here.

http://www.sitemaps.org/protocol.php

Answer 3

Assuming you're trying to create an xml sitemap, not at html one?

I haven't done this yet, and want to try it out soon, but what I would do is: set up a cron job that calls a path like example.com/index.php/sitemap. the index function in your sitemap controller would then run a query to select all titles of all your posts + everything you need to construct a full url to those entries and, since its a required element in the sitemaps protocol, when the entry was last updated. if you don't have a field in your db for this you may have to add it.

then its just a matter of looping through the results and writing out an xml file with the proper structure. as you're looping you'd also have to decide the priority, changefreq and all other elements on a per entry basis.

ive also been looking at the paid version of http://www.xml-sitemaps.com/ which should do everything but setting up the cron for you, at quite a nice price and running on your own server. i dont think you could develop this yourself for 19,99USD worth of your time. And, this app also notifies the search engines of new content, a method for which i havent outlined yet above.

Answer 4

Even though your content is dynamic, you want your sitemap to be static. Sure, update the sitemap once a day if you want, OR update it whenever you post a new blog entry... but don't try to create it at the time that it's requested. That'll be a performance nightmare.

I think Stackoverflow's sitemap is updated daily. It contains the 50,000 most recently changed pages. The sitemap helps with SEO (search engine optimization).

Recipe for sitemap generation over here