import httplib
conn = httplib.HTTPConnection(head)
conn.request("HEAD",tail)
res = conn.getresponse()
print res.status
I am currently using this to get the HTTP header code of a file. However, it seems like this code DOWNLOADS the file, and then gets the code.
However, some files are actually video files...and it would be inefficient for my program to download them.
Is there any way to read the header codes without downloading the file at all?
The purpose of the HTTP HEAD command is to return the header information, identical to what you would receive with a GET command, but without the response body (e.g. the video itself). If you are issuing the HEAD command and receiving the full body of the response anyway, that sounds like a problem with the server you're connecting to.
Unfortunately the HEAD HTTP Method like all other HTTP method is just a directive to the server. The HTTP spec says that the server must not return the body in case, but if the server is not implemented or configured correctly then it may return the entire contents of the URL.
There are other factors that may be at play here a proxy server either on your end or on the server end could be caching the content (especially if since it is video) and returning it from a cache. Since the data is coming from a cache full respect of the HTTP specification may be missing.
As the others have said, if you're getting more than the headers back with the HEAD command, the target server is misconfigured.
However, a pragmatic solution to your problem is to simply ask for the first N bytes of the file, and then parse those as a header. Alternatively, stream the connection, periodically parsing for a complete header, and then cancel the download once you've got the header information you need.
Use urllib2.open() and get the headers already parsed for you, along with a file-handle ready to read the rest of the data stream. At that point you close the file and don't fetch anything more.
>>> import urllib2
>>> f = urllib2.urlopen("http://stackoverflow.com/")
>>> for k,v in f.headers.items():
... print repr(k), "=", repr(v)
...
'content-length' = '113782'
'expires' = 'Tue, 17 Nov 2009 22:12:33 GMT'
'server' = 'Microsoft-IIS/7.0'
'connection' = 'close'
'cache-control' = 'private'
'date' = 'Tue, 17 Nov 2009 22:12:33 GMT'
'content-type' = 'text/html; charset=utf-8'
>>> f.read(20)
'\r\n\r\n<!DOCTYPE HTML P'
>>>
There should be no reason to generate a HEAD request here.