Sitemap.xml - Google not indexing

Question

Hey,

I have created a sitemap for my site and it complies with the protocol set by http://www.sitemaps.org/

Google has been told about this sitemap via webmaster tools. It has tracked all the urls within the sitemap (500+ urls) but has only indexed 1 of them. The last time google downloaded the sitemap was on the 21st of Oct 2009.

When I do a google search for site:url it picks up 2500+ results. Google says it can crawl the site.

Does anyone have any ideas as to why only 1 url is actually indexed?

Cheers, James

Answer 1

First off, make sure Google hasn't been forbidden from those pages using robots.txt, etc. Also make sure those URLs are correct. :)

Second, Google doesn't just take your sitemap at face value. It uses other factors, such as inbound links, etc, to determine whether it wants to crawl all of the pages in your sitemap. The sitemap then serves mostly as a hint more than anything else (it helps Google know when pages are updated more quickly, for example). Get high-quality, relevant, useful links (inbound and outbound) and your site should start getting indexed.

Answer 2

Your two statements seem to contradict one another.

but has only indexed 1 of them.

and

When I do a google search for site:url it picks up 2500+ results

bdonlan is correct in their logic (robot.txt and Google's lack of trust for sitemaps) but I think the issue is what you "think" is true about your site.

That is, Google Webmaster Tools says you only have 1 page indexed but site:yoursite.com shows 2.5k.

Google Webmaster Tools aren't very accurate. They are nice but they are buggy and MIGHT help you learn about issues about your site. Trust the site: command. Your in Google's index if you search site:yoursite.com and you see more than 1 result.

I'd trust site:yoursite.com. You have 2.5k pages in Google, indexed and search-able.

So, now optimize those pages and see the traffic flow. :D

Sidenote: Google can crawl any site, flash, javascript, etc.