In Python (2.5), should I use urllib or urllib2? What's the difference? They seem to do the same thing.
(Bonus points, if I'm using Google App Engine, does this change the answer?)
+2
A:
urllib2.urlopen accepts an instance of the Request class or a url, whereas urllib.urlopen only accepts a url.
A similar discussion took place here: http://www.velocityreviews.com/forums/t326690-urllib-urllib2-what-is-the-difference.html
Danny Roberts
2010-01-07 03:29:44
+1
A:
You should generally use urllib2, since this makes things a bit easier at times by accepting Request objects and will also raise a URLException on protocol errors. With Google App Engine though, you can't use either. You have to use the URL Fetch API that Google provides in its sandboxed Python environment.
Chinmay Kanchi
2010-01-07 03:36:46
What you said about appengine is not entirely true. You can actually use httplib, urllib, and urllib2 in App Engine now (they are wrappers for url fetch, done so that more code would be compatible with appengine.)
Crast
2010-01-07 03:45:25
Ah, must be new. My code failed last I tried and had to be rewritten to work with fetch...
Chinmay Kanchi
2010-01-07 10:30:07
+12
A:
urllib2 provides some extra functionality, namely the urlopen() function can allow you to specify headers (normally you'd have had to use httplib in the past, which is far more verbose. More importantly though, urllib2 provides the Request class, which allows for a more declarative approach to doing a request:
r = Request(url='http://www.mysite.com')
r.add_header('User-Agent', 'awesome fetcher')
r.add_body(urllib.urlencode({'foo': 'bar'})
response = urlopen(r)
Note that urlencode() is only in urllib, not urllib2.
There are also handlers for implementing more advanced URL support in urllib2. The short answer is, unless you're working with legacy code, you probably want to use the URL opener from urllib2, but you still need to import into urllib for some of the utility functions.
Bonus answer With Google App Engine, you can use any of httplib, urllib or urllib2, but all of them are just wrappers for Google's URL Fetch API. That is, you are still subject to the same limitations such as ports, protocols, and the length of the response allowed. You can use the core of the libraries as you would expect for retrieving HTTP URLs, though.
Crast
2010-01-07 03:43:08
How does somebody create a url with an encoded query string using urllib2? It's the only reason I'm using urllib and I'd like to make sure I'm doing everything the latest/greatest way.
Gattster
2010-01-07 08:51:22
Like in my above example, you use `urlopen()` and `Request` from *urllib2*, and you use `urlencode()` from *urllib*. No real harm in using both libraries, as long as you make sure you use the correct urlopen. The [urllib docs][1] are clear on that using this is acecepted usage. [1]: http://docs.python.org/library/urllib2.html#urllib2.urlopen
Crast
2010-01-07 14:12:33
+1
A:
I like the urllib.urlencode function, and it doesn't appear to exist in urllib2.
>>> urllib.urlencode({'abc':'d f', 'def': '-!2'})
'abc=d+f&def=-%212'
Gattster
2010-01-07 03:51:08