How to check if a webpage exists. jQuery and/or PHP

Question

I want to be able to validate a form to check if a website/webpage exists. If it returns a 404 error then that definitely shouldn't validate. If there is a redirect...I'm open to suggestions, sometimes redirects go to an error page or homepage, sometimes they go to the page you were looking for, so I don't know. Perhaps for a redirect there could be a special notice that suggests the destination address to the user.

The best thing I found so far was like this:

$.ajax({url: webpage ,type:'HEAD',error:function(){
    alert('No go.');
}});

That has no problem with 404's and 200's but if you do something like 'http://xyz' for the url it just hangs. Also 302 and the like trigger the error handler too.

This is a generic enough question I would like a complete working code example if somebody can make one. This could be handy for lots of people to use.

Answer 1

First you need to check that the page exists via DNS. That's why you say it "just hangs" - it's waiting for the DNS query to time out. It's not actually hung.

After checking DNS, check that you can connect to the server. This is another long timeout if you're not careful.

Finally, perform the HTTP HEAD and check the status code. There are many, many, many special cases you have to consider here: what does a "temporary internal server error" mean for the page existing? What about "permanently moved"? Look into HTTP status codes.

Answer 2

Maybe you could combine domain checker, and jQuery, domain checker (PHP) can respond 1 or 0 for non-existent domains.

eg. http://webarto.com/snajper.php?domena=stackoverflow.com , will return 1, you can use input blur function to check for it instantly.

Answer 3

It sounds like you don't care about the web page's contents, you just want to see if it exists. Here's how I'd do it in PHP - I can stop PHP from taking up memory with the page's contents.

/*
 * Returns false if the page could not be retrieved (ie., no 2xx or 3xx HTTP
 * status code). On success, if $includeContents = false (default), then we
 * return true - if it's true, then we return file_get_contents()'s result (a
 * string of page content).
 */
function getURL($url, $includeContents = false)
{
  if($includeContents)
    return @file_get_contents($url);

  return (@file_get_contents($url, null, null, 0, 0) !== false);
}

For less verbosity, replace the above function's contents with this.

return ($includeContents) ? 
               @file_get_contents($url) :  
               (@file_get_contents($url, null, null, 0, 0) !== false)
;

See http://www.php.net/file_get_contents for details on how to specify HTTP headers using a stream context.

Cheers.