Why aren't persistent connections supported by URLLib2?

views:

answers:

+3 Q:

Why aren't persistent connections supported by URLLib2?

After scanning the urllib2 source, it seems that connections are automatically closed even if you do specify keep-alive.

Why is this?

As it is now I just use httplib for my persistent connections... but wonder why this is disabled (or maybe just ambiguous) in urllib2.

+2 A:

It's a well-known limit of urllib2 (and urllib as well). IMHO the best attempt so far to fix it and make it right is Garry Bodsworth's coda_network for Python 2.6 or 2.7 -- replacement, patched versions of urllib2 (and some other modules) to support keep-alive (and a bunch of other smaller but quite welcome fixes).

Alex Martelli 2010-09-15 23:33:08

Alex, finally a straight answer on this one ('its' a well known limit'), The question remains though, why is URLLib2 written this way?

sbartell 2010-09-17 17:44:47

@sbartell, because nobody felt the problem was important enough to submit a patch to the Python code and have it accepted -- I didn't, neither did you, and so on for millions of people who could and no doubt would **if** they felt the problem was important (assuming they're decent citizens of the open-source community, of course, but, hey, aren't we all?-). I think Gary took the right approach by releasing a third-party solution so that lots of real-world "field" experience in a variety of uses can be accumulated before things are "frozen" into the standard library.

Alex Martelli 2010-09-17 18:09:57

You might also check out httplib2, which supports persistent connections. Not quite the same as urllib2 (in the sense that it only does http and not "any kind of url"), but easier than httplib (and imho also easier than urllib2 if you really want to do http).

Steven 2010-09-16 08:34:28

httplib does support them, we just use the same httpconnection object over again.

sbartell 2010-09-17 17:46:00

It just boggles me why urllib2 does support this.

sbartell 2010-09-17 17:46:48

ansaurus

tags:

views:

answers:

Why aren't persistent connections supported by URLLib2?

related questions