Is there an easy way to request a URL in python and NOT follow redirects?

Question

Looking at the source of urllib2 it looks like the easiest way to do it would be to subclass HTTPRedirectHandler and then use build_opener to override the default HTTPRedirectHandler, but this seems like a lot of (relatively complicated) work to do what seems like it should be pretty simple.

Answer 1

Dive Into Python has a good chapter on handling redirects with urllib2. Another solution is httplib.

>>> import httplib
>>> conn = httplib.HTTPConnection("www.bogosoft.com")
>>> conn.request("GET", "")
>>> r1 = conn.getresponse()
>>> print r1.status, r1.reason
301 Moved Permanently

Answer 2

I second olt's pointer to Dive into Python. Here's an implementation using urllib2 redirect handlers, more work than it should be? Maybe, shrug.

import sys
import urllib2

class RedirectHandler(urllib2.HTTPRedirectHandler):
 def http_error_301(self, req, fp, code, msg, headers):  
  result = urllib2.HTTPRedirectHandler.http_error_301( 
   self, req, fp, code, msg, headers)              
  result.status = code                                 
  raise Exception("Permanent Redirect: %s" % 301)

 def http_error_302(self, req, fp, code, msg, headers):
  result = urllib2.HTTPRedirectHandler.http_error_302(
   self, req, fp, code, msg, headers)              
  result.status = code                                
  raise Exception("Temporary Redirect: %s" % 302)

def main(script_name, url):
   opener = urllib2.build_opener(RedirectHandler)
   urllib2.install_opener(opener)
   print urllib2.urlopen(url).read()

if __name__ == "__main__":
 main(*sys.argv)

Answer 3

i suppose this would help

from httplib2 import Http
def get_html(uri,num_redirections=0): # put it as 0 for not to follow redirects
conn = Http()
return conn.request(uri,redirections=num_redirections)