Google and Mirror Websites

Question

Which is the best way to manage a website with one or more mirrors so that:

• Google don't consider it as "dupicated content"
• The website is correctly indexed
• No inconsistencies or duplicated information are present in Google Analytics
• The Google webmaster guidelines in general are respected

NOTE: I'm not sure if I should ask this question here or in ServerFault. It looks a bit in the middle between programming and server administration. Let me know if you think ServerFault represent a more appropriate place for this and I'll move it.

Thanks.

Answer 1

The usual approach is to have Google index the master site and then install a round robin plugin on this server which sends other visitors to one of the mirrors. That will distribute the load and Google will have to index just a single site.

Answer 2

The official and simple solution is the canonical link tag. This is the official solution recommeded by Google.

http://googlewebmastercentral.blogspot.com/2009/02/specify-your-canonical.html

Answer 3

If you need to host on multiple servers using round-robin DNS (or other load-balancing techniques) is a good idea. This will let you use a single host name and would generally not create problems with crawling and indexing on the search engine side (since the crawlers don't see multiple URLs for the content).

If you need to host using separate host or domain names (for whatever reason) it's best to pick one preferred version and to make sure that only that one is indexed. A way to do that could be to use rel=canonical link elements on the alternate versions. In general, however, I'd recommend working to prevent multiple host/domain names from being visible to the user & search engines by keeping the technical hosting issues (mirrored hosts) out of sight (as mentioned in the first part).

If you need to use multiple ccTLDs to host on country-specific domains then I'd strongly recommend making sure that you actually have country-specific content on each site (and not just mirroring one version). More about this is at http://googlewebmastercentral.blogspot.com/2010/03/working-with-multi-regional-websites.html