Using urllib2 for posting data, following redirects and maintaining cookies

Question

I am using urllib2 in Python to post login data to a web site.

After successful login, the site redirects my request to another page. Can someone provide a simple code sample on how to do this in Python with urllib2? I guess I will need cookies also to be logged in when I get redirected to another page. Right?

Thanks a lot in advace.

Answer 1

First, get mechanize: http://wwwsearch.sourceforge.net/mechanize/
You could do this kind of stuff with just urllib2, but you will be writing tons of boilerplate code, and it will be buggy.

Then:

import mechanize

br = mechanize.Browser()
br.open('http://somesite.com/account/signin/')

br.select_form('loginForm')    
br['username'] = 'jekyll'
br['password'] = 'bananas'
br.submit()
# At this point, you're logged in, redirected, and the 
#  br object has the cookies and all that.

br.geturl() # e.g. http://somesite.com/loggedin/

Then you can use the Browser object br and do whatever you have to do, click on links, etc. Check the samples on the mechanize site