How best to validate that a URL is on the public internet?

Question

I want to validate that a hostname/IP address is on the public internet; that is, that as far as is reasonable, I'd get the same response from it no matter where I access it from (obviously that's impossible to guarantee.

ie I want to exclude localhost, 127.0.0.1, anything in the private IP ranges, and anything that has an invalid TLD.

Am I missing anything else that I ought to be checking?

And is there a better list than http://data.iana.org/TLD/tlds-alpha-by-domain.txt for a list of valid TLDs?

Answer 1

A valid TLD may still resolve to a local address if you do not have strict control over the DNS or /etc/hosts, so resolving and then excluding by IP range (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 169.254.0.0/16, 127.0.0.0/8) is best.

Your TLD list is up-to-date.

Answer 2

IANA is the official source for information on domain names, so you can't get a better list - or at least, you can't get any more authoritative.

Answer 3

Wouldn't the ultimate validation be to create a list of networks local to you (for instance, behind your own firewall) and, if it's not, try to connect the host? If you can connect and it's not local, you would have no reason to expect that any other location on the internet couldn't connect.