Sitemap for a site with a large number of dynamic subdomains

Question

Hi,

I'm running a site which allows users to create subdomains. I'd like to submit these user subdomains to search engines via sitemaps. However, according to the sitemaps protocol (and Google Webmaster Tools), a single sitemap can include URLs from a single host only.

What is the best approach?

At the moment I've the following structure:

1. Sitemap index located at example.com/sitemap-index.xml that lists sitemaps for each subdomain (but located at the same host).
2. Each subdomain has its own sitemap located at example.com/sitemap-subdomain.xml (this way the sitemap index includes URLs from a single host only).
3. A sitemap for a subdomain contains URLs from the subdomain only, i.e., subdomain.example.com/*
4. Each subdomain has subdomain.example.com/robots.txt file:

--

User-agent: *
Allow: /

Sitemap: http://example.com/sitemap-subdomain.xml

--

I think this approach complies to the sitemaps protocol, however, Google Webmaster Tools give errors for subdomain sitemaps: "URL not allowed. This url is not allowed for a Sitemap at this location."

I've also checked how other sites do it. Eventbrite, for instance, produces sitemaps that contain URLs from multiple subdomains (e.g., see http://www.eventbrite.com/events01.xml.gz). This, however, does not comply with the sitemaps protocol.

What approach do you recommend for sitemaps?

Answer 1

hi, ok

yes, the subdomain restriction is in the sitemaps.org spec.

but:

google has put some exceptions in place

1) verify all subdomains within your google webmaster tools account http://www.google.com/support/webmasters/bin/answer.py?answer=75712 cross-submission of sitemaps XML via google wembaster tools - if submitted via the root of your domain - will not throw errors for google

2) within the robots.txt of a subdomain you can point to sitemaps XML on other domains. there will be no cross submission errors - for google

Answer 2

Yes it is restricted. You need to create different sitemaps for each. To do that you can use online sitemap tool