views:

596

answers:

4

Is there a way to stop Google from indexing a site?

+3  A: 

Google obeys the robots.txt file.

tvanfosson
What is the robots.txt file?
Nick
Added link to the wikipedia article on robots.txt
tvanfosson
+5  A: 

http://www.robotstxt.org/

Ruggs
+17  A: 

robots.txt

User-agent: *
Disallow: /

this will block all search bots from indexing.

for more info see: http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=40360

Unkwntech
Actually, to be precise, this will block all *legitimate* bots from crawling the site. Malicious ones will still attempt to do so, just in case that matters.
Software Monkey
That is correct however if the "spider" does not check robots.txt then it is likely malicious, which from my experience means that they will also spoof the user-agent which makes it ridiculously hard to stop.
Unkwntech
+4  A: 

FYI - Google has a Webmaster Tools site that is worth at least checking out .. .. http://www.google.com/webmasters/start/

Scott Evernden