Many web sites know where I'm from. For example, visiting Yelp shows me restaurant listings from the nearest metropolitan area and searching for "maps" on Live Search shows me a map of my current city.
My best guess is sites do this by using a large database mapping the user's IP address to city name. If this is true, where's the best place to obtain such a database at a reasonable cost? A Google search matches a lot of random vendors I've never heard of.
There is a (very) old free database at netgeo, which also lists some alternatives.
ip2location offers a demo service of 20 lookups a day you could play with
I work (tangentially) with the two largest ISPs in the U.S. and can tell you that when you choose a vendor, you'll need a long term relationship. These companies move blocks of IP addresses around fairly frequently, and you'll need a subscription to the data so that you are always up-to-date. There is good news coming though (maybe a long time from now) ... IPv6 will make this much easier!
AdultFriendFinder's geolocated banner ads are hackable if you want to try the ghetto route; msteigerwalt has the details.
Visit a demo site that uses Google's AJAX apis.
In firebug enter google.loader.ClientLocation.address.city into the console. That's Google ClientLocation for you, however it only works well in a few places in the world.