I have extensive experience with PHP cURL but for the last few months I've been coding primarily in Java, utilizing the HttpClient library.
My new project requires me to use Python, once again putting me at the crossroads of seemingly comparable libraries: pycurl and urllib2.
Putting aside my previous experience with PHP cURL, what is the recommended library in Python? Is there a reason to use one but not the other? Which is the more popular option?
urllib2 is part of the standard library, pycurl isn't (so it requires a separate step of download/install/package etc). That alone, quite apart from any difference in intrinsic quality, is guaranteed to make urllib2 more popular (and can be a pretty good pragmatical reason to pick it -- convenience!-).
CURL has a lot more features as stated in its web page, so if you need, say fast concurrent connections, safe threading, etc then its for you. However, its not included in the distribution. If you foresee that your task is very simple, then use urllib2 and those HTTP modules that come with the distribution.
Use urllib2. It's got very good documentation in python, while pycurl is mostly C documentation. If you hit a wall, switch to mechanize or pycurl.
I'm recently using urllib2 and getting hit by some strange cookie and redirection problems. Don't know whether other python lib could help me solve the problems yet.