noindex, noarchive for dead web pages? what do you think is correct?

Question

I've noticed that a fair number of web apps, in particular do not deal very well with pages/accounts that have been deleted.

First off, I'll state that I am of the position that the content owner always owns the content and that if the content is deleted or the owner deletes the account, the provider/web app should do everything possible to stop indexing of said content.

To that, I would think that a reasonable strategy would be to set 404 pages and placeholder pages to noindex, nofollow and noarchive in their meta tags.

So a couple example cases, Flickr does not do either when an account is deleted, instead it returns page saying account is deleted.

www.flickr.com/people/rebelchrome

Friendfeed returns a 404 with no special meta tags.

What do you think is the best/right to do in situations like this?

Answer 1

The response code 410 Gone is for dead web pages (web pages which no longer exist and for which there is no obvious alternative). The page can still return a body.

Search engines encountering a 410 Gone status response will be able to realise that the page no longer exists and can act accordingly - for most search engines, this would mean simply taking it out of their index.

Humans encountering the page will just see the page body. Just like with a 404, you can have a custom 410 page, and this could be similar - containing a brief message that the page no longer exists, and maybe also a mini site map and search box allowing the user to find alternative content on the site.

Using robots instructions like noindex are not really necessary when the page returns a 410/404 response, because the response code says it all, really.

The flickr page you linked to has the message, mini site map and search box, but probably should return a 410 or 404 error response, not a 200 response as it does.