Web automation

Question

I'm developing an interface between an old web based application and another one. That old web-based application works fine but there no exists any API to communicate with.

There is any programmatic way to say a web-form something like: enter this value on this field, this one ins other and submit form?

UPDATE: I looking for something like this:

WebAutomation w = new WebAutomation("http://apphost/report");
w.forms[0].input[3].value = 123;
w.forms[0].input[4].value = "hello";
Response r = w.forms[0].submit();
...

Answer 1

In .NET: http://watin.sourceforge.net/

In ruby: http://wtr.rubyforge.org/

Cross platform: http://seleniumhq.org/

Answer 2

You can, but you have to mock up a POST request. The fields (textboxes, radio buttons, etc.) are transmitted as key-value pairs back to the resource. You need to make a request for this resource(whichever one is used in the SUBMIT action for the FORM tag) and put all your field-value pairs in a POST payload no the request. Here's a good program to see what values are being transmitted: http://www.httpwatch.com Or, you can use Firebug, a free Firefox extension.

Answer 3

Despite the tag on your question, the answer is going to be highly language specific. There are also going to be wide range of solutions depending on how complex of a solution you are willing to implement and how flexible a result you are looking for.

On the one hand you can accomplish a lot in a very short period of time with something like Python's mechanize, but on the other hand, you can really get into the guts and have a lot of control by automating a browser using a COM object such as SHDocVw (Windows-only, of course).

Or, as LoveMeSomeCode suggested, you can really hit your head against the concrete and start forging POST requests, but good-luck figuring out what the server expects if is doing any front-end processing of the form data.

EDIT:

One more option, if you are looking for something that you can come up to speed on quickly, is to use a AutoIt's IE module, which basically provides a programmatic interface over an instance of Internet Explorer (its all COM in underneath, of course). Keep in mind that this will likely be the least supportable option you could choose. I have personally used this to produce proof-of-concept automation suites that were then migrated to a more robust C# implementation where I handled the COM calls myself.

Answer 4

The Perl module WWW::Mechanize does exactly that. Your example would look something like this:

use WWW::Mechanize;
my $agent = WWW::Mechanize->new;
$agent->get("http://apphost/report");
my $response = $agent->submit_form(
    with_fields => {
        field_1_name => 123,
        field_2_name => "hello",
    },
);

There is also a Python port, and I guess similar libraries exist for many other languages.