What technical features could I include in my 404 File Not Found error handling page?

Question

I have a custom 404 error handling page

It looks up the URL in a database table and redirects if there is a suitable target. URLs are logged, together with the Redirect provided (if any) and our App has an Admin report to show which URLs are getting caught, allowing Client to set up more etc.

We have a single, specific, Images folder, but we get image requests with malformed paths. Where we can find a matching image name in /IMAGES/ we return that. Should I use a 301? (we currently return a 200)

We highlight these on the Admin report - because likelihood is that there is an error in CMS or a bulk Email or somesuch, and by redirecting we are just masking the problem (and fixing it will help performance I think?)

I am wondering if we should return a dummy image when we get a 404 on a missing JPG/GIF/PNG? Currently we are returning 404 result and a sorry page in HTML - which strikes me as a bit daft, would the user's browser do anything useful with a returned Image if there is a 404 response code?

I also wonder if returning an image "Image not found, visit www.example.com" would be helpful (perhaps specifically if my domain is NOT the referrer!). Then the useless person who embeds our images in their site, wrongly at that!, might at least drive us some traffic.

Similarly should I return something useful if I get a 404 request for JS or CSS files? I'm thinking in DEV, at least, it would be handy to know we've goofed up. Sometimes the missing file can be sufficiently obscure, in its usage, that its absence is missed in QA. (I suppose someone OUGHT to notice it in the 404 logs!), but I'm thinking of maybe setting BODY to something massive, or an ALERT in the .JS file returned, might help in DEV.

In Googling around this today I also fell over the suggestion that a malformed query string could return a "400 Bad Request" and a well formed query string, but where a parameter has an invalid value (e.g. product code not found) could be treated as a 404. If I do that and also return content (e.g. an explanation page) will the user see that, or might their browser replace it with a canned 404-error page? (I had a feeling that an earlier version of IE did that?)

All ideas appreciated.

(Classic ASP/IIS in my case, but hopefully the question is generic)

Edit: I also wonder if anyone does anythign special with things that look like known hack attempts?

• http://www.example.com:80/admin/phpmyadmin/scripts/setup.php
• http://www.example.com:80/admin/pma/scripts/setup.php
• http://www.example.com:80/admin/scripts/setup.php
• http://www.example.com:80/db/scripts/setup.php
• http://www.example.com:80/dbadmin/scripts/setup.php
• http://www.example.com:80/myadmin/scripts/setup.php
• http://www.example.com:80/mysql/scripts/setup.php
• http://www.example.com:80/mysqladmin/scripts/setup.php
• http://www.example.com:80/phpadmin/scripts/setup.php
• http://www.example.com:80/phpMyAdmin/scripts/setup.php
• http://www.example.com:80/phpmyadmin1/scripts/setup.php
• http://www.example.com:80/phpmyadmin2/scripts/setup.php
• http://www.example.com:80/pma/scripts/setup.php
• http://www.example.com:80/web/scripts/setup.php

and these "feelers":

• http://www.example.com:80/_vti_bin/owssvr.dll?...
• http://www.example.com:80/MSOffice/cltreq.asp?...

Edit2: Sorry, hopefully the last after-though.

Should I allocate a Session ID? That would enable me to track whether the user comes back with something more intelligent at a second attempt (which might cause us to add an entry to our Redirect table). Creating a session involves creating a Session record in the database and some other stuff, so is not as "cheap" as just giving out a 404 error

Answer 1

I once made a web page that dynamically generated images. The problem was how to propagate errors when the image could not be rendered.

The solution, which I think is quite good, was simply to print the error message in the image itself. Then you could HREF the image and if it didn't work, you'd get the error message printed out. Better than just returning an invalid image.

For JS-files, you could do the same, but I don't know if it would be a good design. I don't think even returning an empty text document would be good in that case. You could return error messages in alert-boxes, and hook them into your main application, but I guess that's more useful for developers, and not general users. (Depends who your users are). I think the same goes for CSS.

For error messages in general, if you just want to know that an invalid URL was reached, you could either search the server logs, or you could hide a script behind the JS/CSS URLs that logged an informative message somewhere.

Lastly, remember that the HTTP codes are made for a reason. You can expect the clients to handle valid HTTP error codes better than "faked" and seemingly good output (from an HTTP perspective). In other words, you could use this to your advantage and respond with more specific HTTP error codes in situations where the normal ones are not adequate.