How can urllib2 / httplib talk HTTP 1.1 for HTTPS connections via a Squid proxy ?

Question

When I use urllib2 to make a HTTP 1.1 connection via a squid proxy, squid makes a new ongoing connection in HTTP 1.0.

How can I persuade Squid to talk 1.1 to the destination server?

Answer 1

After dealing with this problem for an entire afternoon, i found the solution. So please excuse me answering my own question, but it would be great if someone else finds this useful and it saves them the pain.

In order to get Squid to have a HTTP 1.1 conversation with the destination server, the original request to it must be done via HTTP CONNECT. This is documented in the bug http://bugs.python.org/issue1424152.

There is a fix for py3k and it has been backported to Python 3.1 and 2.6.

If you are rocking a Python 2.5 or 2.4 installation, then you can download a patched version of httplib.py and urllib2.py here http://pypi.python.org/pypi/httpsproxy%5Furllib2. Simply replace your existing versions, or drop these 2 files into your project.