I'm developing a web site with Google App Engine, and I want to have a search feature for user submitted stuff. Since this project is just a toy and I don't control the server, I'd like to just use Google to handle search. However, since the content is stored in the database, I don't think Google can discover the dynamic urls. Unless maybe I create a page that links the last N submissions and hope it gets crawled frequently. Thoughts?
+2
A:
Absolutely. As long as the database is exposed in a web page which can be crawled, Google will crawl it (unless told not to).
The best way to make it all accessible is decent navigation between pages. However, lacking that, a site map page linked from the home page should suffice.
wallyk
2010-01-12 00:24:18
Steve Jessop
2010-01-12 00:31:09
A:
This is an excellent candidate for a sitemap.
You can generate the XML any way you want, and give it to Google. The best part is, it is a "private" XML file; no need to have ugly listings of dynamic URLs for users to see.
jhs
2010-01-12 02:10:42
One question: Can I resubmit the sitemap with only new URLs in it, thus avoiding the need to keep multiple sitemaps with every link ever added? I.E. will it remember the URLs from previous submissions, so they don't need to be listed again?
Anon
2010-01-12 17:56:16