How to submit web forms using Python?

Question

First of all, sorry if this question is a little vague and rambling! I'm ok with Python, but I've never done anything HTTP related before.

I'm trying to automate submitting a web form, and from reading some of this page I understand that I need to do a POST request. I also found a code snippet demonstrating the urllib module:

import urllib
params = urllib.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0})
f = urllib.urlopen("http://www.musi-cal.com/cgi-bin/query", params)
print f.read()

But I still don't really understand what I'm doing. I need to trigger "submit" somehow, and I assume the actual data I'm submitting will go in the params somewhere?

Answer 1

The code there should do what you want.

Whatever data you want to use should go into the params as you have in your example. When the params are included as an argument to urlopen a POST request will be used (instead of a GET).

By just calling urlopen I believe the POST request will be submitted. If you want the response however you will need to use f.read().