Would 401 Error be a good choice?

Question

Hello,

On one of my sites have a lot of restricted pages which is only available to logged-in users, and for everyone else it outputs a default "you have to be logged in ... " view.

The problem is; a lot of these pages are listed on Google with the not-logged-in-view, and it looks pretty bad when 80% of the pages in the list have the same title and description/preview.

Would it be a good choice to, along with my default not-logged-in-view, send a 401 unauthorized header? And would this stop Google (and other engines) to index these pages?

Thanks!

(and if you have another (better?) solution I would love to hear about it!)

Answer 1

Use a robots.txt to tell search engines not to index the not logged in pages.

http://www.robotstxt.org/

Ex.

User-agent: *
Disallow: /error/notloggedin.html

Answer 2

Response code 403 is for requests, where authentication makes no difference, eg. disabled directory browsing.
401 Unauthorized would be the response to send.

Answer 3

hi, here are the status codes googlebot understands and recommends. http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=40132 in your case an HTTP 403 would be the right one.