How do I keep Google from indexing my custom 404 error page?

Question

I use ASP.Net and I have a custom 404 page. When user pastes url that is not found it will redirected to the custom 404 page. However google indexes my custom 404 page. Search 404(page not found).

Anyone have solution?

Answer 1

You will always see a log entry for Google hitting your 404 page. Because Google needs to request the page in order to know that it has a status of 404. However you should never see this page in the search results, if that is what you are asking?

Answer 2

You can make your errors directory or each file off-limits to google by adding it to your robots.txt file in the root of your site. This actually applies to any directories/files that you don't want google to index.

More information on robots.txt can be found at http://en.wikipedia.org/wiki/Robots.txt or http://www.robotstxt.org/.

Answer 3

You can use a robots meta tag to stop Google from indexing the 404 page.

<META NAME="ROBOTS" CONTENT="NOINDEX">

The Google webmaster blog indicates that they won't index pages with this meta tag.

Answer 4

You may want to make sure your "404 page" is actually sending an HTTP 404 response code, rather than just a page that says 404 but actually responds with HTTP 200. I don't believe Google (or other major search engines) will index pages resulting from requests that respond with 404.

The bigger question is how a spider is getting to your 404 page.