How do I log in to a site remotely using a script?

Question

Hi all,

I'm trying to write a script to automate a repetitive task I currently do manually. I would like to log in to a site remotely and inspect the returned page.

I've been doing this by sending a direct POST request (the site is PHP, I'm pretty sure it's Joomla) with my login details and data for the other fields of the form from the front page, but I'm getting either sockaddrinfo errors on the Net:HTTP Ruby library when I try a HTTP.post() (with data as a param=val1&param2=val2 string), and a rejected redirect to home page if I use HTTP.post_form (using a Hash)

I'm willing to do this in any language, really, I just picked Ruby since it's a favorite for quick scripting. If you have any ideas in bash, Python, etc. I'd be happy to try it.

I've tried variations on some examples, to no avail. Have any of you tried something like this with success? Any stumbling blocks we beginners run into frequently?

Thanks for your time ^_^

-Paul

Answer 1

Try mechanize:

http://mechanize.rubyforge.org/mechanize/EXAMPLES_rdoc.html

Answer 2

Have a look at mechanize (Python) which is written with your problem in mind:

import re
from mechanize import Browser

br = Browser()
br.open("http://www.example.com/")
# follow second link with element text matching regular expression
response1 = br.follow_link(text_regex=r"cheese\s*shop", nr=1)