I'm building a web portal where language content will generally depend on the "accept-language" sent by the browser. The same content-URI will thus serve different content to different users depending on their browser setting.
I'm very curious to know how this will affect search indexing. Does Google index using all languages, and is it handled well?
Kien Problem!
Google doesnt care what language the text is in it just indexes!
Probably the Googlebot doesn't send any accept-language. I would program the site to spit all-information, in all languages if no accept-language is sent.
Another solution is to have links that change the language by adding a URL query but not changing the base part of the URL, so Googlebot will still traverse all languages and point people to the right address. Having links to change language is also useful for the cases when the user actually wants to do it.
Disclaimer: I work for Google but I don't have a clue about the Googlebot.
The accept-language HTTP header is not mandatory, so you should provide a default for when the user agent doesn't send one.
The Googlebot which indexes pages for Google will probably not send an accept-language, since it is interested in all languages and doesn't want to be constrained to certain ones.
They don't send accept-language, so the site will be indexed in the default language that you select.
I recommend you to have different URL for each language, not only for the search engines, but for letting the user change the site language without changing the "accept-language" and letting the user to send a URL to a friend with the language selected.
In this answer you have how I managed it in .net, but you will get the idea: Using the "accept-language" the make the first redirect, and them play only with the url.
This I asked myself exactly. According to the W3C this is for sure possible and we can display a website in different languages. We can also display a menu with other languages and store this in a cookie for the user so we can perfectly display a site in different languages for users.
So when a user is logged in we show default english, but he can pick from a menu a other language. We store this in a cookie and off we go. We CAN show all URL's in different languages.
However search engine breaks this possibility and INSIST we have a different URL for each language, and punish US for showing duplicate content!!! (Although the language is different the content/message is the same.... right???)
This method of working with search engines basicly suck. Although I also understand that WE (webdesigners) don't have a good option to tell a client in what languages a specific URL is available in (except using different URL's)
Ries vanTwisk