views:

1854

answers:

3

Looking at the source of urllib2 it looks like the easiest way to do it would be to subclass HTTPRedirectHandler and then use build_opener to override the default HTTPRedirectHandler, but this seems like a lot of (relatively complicated) work to do what seems like it should be pretty simple.

+13  A: 

Dive Into Python has a good chapter on handling redirects with urllib2. Another solution is httplib.

>>> import httplib
>>> conn = httplib.HTTPConnection("www.bogosoft.com")
>>> conn.request("GET", "")
>>> r1 = conn.getresponse()
>>> print r1.status, r1.reason
301 Moved Permanently
olt
+3  A: 

I second olt's pointer to Dive into Python. Here's an implementation using urllib2 redirect handlers, more work than it should be? Maybe, shrug.

import sys
import urllib2

class RedirectHandler(urllib2.HTTPRedirectHandler):
 def http_error_301(self, req, fp, code, msg, headers):  
  result = urllib2.HTTPRedirectHandler.http_error_301( 
   self, req, fp, code, msg, headers)              
  result.status = code                                 
  raise Exception("Permanent Redirect: %s" % 301)

 def http_error_302(self, req, fp, code, msg, headers):
  result = urllib2.HTTPRedirectHandler.http_error_302(
   self, req, fp, code, msg, headers)              
  result.status = code                                
  raise Exception("Temporary Redirect: %s" % 302)

def main(script_name, url):
   opener = urllib2.build_opener(RedirectHandler)
   urllib2.install_opener(opener)
   print urllib2.urlopen(url).read()

if __name__ == "__main__":
 main(*sys.argv)
Aaron Maenpaa
+5  A: 

i suppose this would help

from httplib2 import Http
def get_html(uri,num_redirections=0): # put it as 0 for not to follow redirects
conn = Http()
return conn.request(uri,redirections=num_redirections)
Ashish