httplib in Python to get the status code...but it is too tricky?

Question

>>> import httplib
>>> conn = httplib.HTTPConnection("www.google.com")
>>> conn.request("HEAD", "/index.html")
>>> res = conn.getresponse()
>>> print res.status, res.reason
200 OK

This code will get the HTTP status code. However, notice that I split up "google.com" and "/index.html" on 2 lines.

And it's confusing.

What if I want to find the status code of just a general URL???

http://mydomain.com/sunny/boo.avi
http://anotherdomain.com/podcast.mp3
http://anotherdomain.com/rss/fee.xml

Can't I just stick the URL into it, and make it work?

Edit...I cannot use urllib, because I don't want to downlaod the file

Answer 1

According to the spec you're supposed to split it up like that, maybe Python could abstract that out for you a bit, they're probably just giving you straight access to the header so you know exactly how it's being formatted, which is really the preference.

Answer 2

Maybe you are better off using the URL library instead?

In Python 2, use urllib2:

>>> import urllib2
>>> url = urllib2.urlopen("http://www.google.com/index.html")
>>> url.getcode()
200

In Python 3, use urllib.request:

>>> import urllib.request
>>> url = urllib.request.urlopen("http://www.google.com/index.html")
>>> url.getcode()
200

Answer 3

The connect method takes a server argument (with an optional port). You have to split the connection with the resource you actually want.

For a simpler way to download web resources directly, you could go with urllib2 but urllib2 only supports GET or POST methods, no HEAD, so you end up downloading the whole resource.

Answer 4

I like urllib2, sample code:

import urllib2
res = urllib2.urlopen('http://google.com/index.html')
res.getCode() #contains code

I something went wrong, you'll get an exception you can catch.

EDIT: Thanks, changes res.code to res.getCode() since the second one is documented

Answer 5

Alternatively, if you expect that actually downloading the data is problematic and you really need the HEAD method, you could parse the URL using urlparse:

>>> import httplib
>>> import urlparse
>>> url = "http://www.google.com/index.html"
>>> (scheme, netloc, path, params, query, fragment) = urlparse.urlparse(url)
>>> conn = httplib.HTTPConnection(netloc)
>>> conn.request("HEAD", urlparse.urlunparse(('', '', path, params, query, fragment))
>>> res = conn.getresponse()
>>> print res.status, res.reason
302 Found

And wrap this into a function taking the URL as an argument.

Answer 6

Keep in mind that not all web servers support HEAD on each resource so you'll end up retrieving the resource anyway. You should write code accordingly.