Why does a Non-existant page returns 302 status when using a custom 404 page in asp.net

Question

I have setup a custom 404 page custom404.aspx that returns a "404 Not Found" error correctly, however the non-existant page that was initially requested returns a "302 Found" status.

So when I test thispagedoesnotexist.aspx, it returns a "302 Found" then the custom404.aspx loads and returns a "404 Not Found" status.

I want to make sure that search spiders/bots understand that the requested page does not exist and should not show up in any search results. Is this setup properly?

Answer 1

Simply put ...

As an example, use:

/path_to_error_page.html 

... and not ...

http://www.example.com/path_to_error_page.html

The reason being that the server interprets the initial request then generates a redirect to the 404 so your client effectively gets 2 responses.

Relative paths don't produce a redirect but an internal server transfer this results in only 1 response, the one you want !!

This should fix your problem.

Answer 2

To answer the stated question, that's just how the custom errors work in ASP.NET. I've seen examples (though I don't have one handy) where people have used HttpModules to intercept errors, and serve a custom error page to users that don't appear to be bots, and a 404 response to users that appear to be bots.

In a different direction, I'm not sure bots regularly follow 302 responses anyway. More relevant perhaps, is it really so bad to have your error page (assuming it's usable) show up in search results? At least the user has a link to your site--that's better than having your competitors site show up...

Answer 3

Actually, it's most likely working exactly as it's configured in your web.config.

Depending on what web server you use (IIS 6/7) and which way you used to configure the 404 custom page, but basically this is the difference between redirecting to the custom error page, and executing it!

Currently you have configured the asp.net / IIS to redirect to the error page. Change the settings to execute the page and you'll get exactly what you're looking for :)

Answer 4

I have found a solution for that:

in your web.config put "redirectMode=ResponseRewrite" :
<customErrors mode="On" redirectMode="ResponseRewrite" defaultRedirect="~/errors/GeneralError.aspx"> <error statusCode="404" redirect="~/errors/PageNotFound.aspx" /> </customErrors>

... And in your custom 404 page in Page_Load put : this.Response.Status = "404 Not Found";

Voila!