Automated tool/APIs to fill up web forms

Question

I am looking for a tool or API/library to automatically fill up a webform with data. I have a text file with 100s of records that I need to enter into a webform. I tried searching for some tool or browser APIs so that I can copy text to a clipboard and from the clipboard to the webform; but the search results are polluted by automated Web Form management tools. I looked at selenium but it seems oriented towards testing. i.e. it records the data entry process and plays it back again, This wont work for me.

Answer 1

Why would you want to fill the form instead of writing the values directly to the database? In what form is the source data available? Is this a repetitive process, how much of it should be automated, how much hand work?

Answer 2

If the data filled in form will goes to database after submitting it, then instead of manually filling form or using any automated tools, it is better to create a script which will read text file and make insertion operation in database. but for this you need database access, if you don't then may be you are trying to make some unethical operations on third party site.

Answer 3

Since you're asking on SO I will assume that you can write code. Personally I would recommend Selenium RC (not the browser plugin, the remote control) for this. It takes a bit of work to set up but not much and the end result is that you will have a system where it is easy to programmatically interact with websites.

Here's an example in Perl:

use Test::WWW::Selenium;

my $sel = Test::WWW::Selenium->new(
    host => 'localhost',
    port => 4444,
    browser => '*firefox',
    browser_url => 'http://your_site.com'
);
$sel->start;
$sel->open('/page/to/test.cgi');

# open file to process:
open(FH, '<', 'path/to/datafile.txt');
while (<FH>) {
    @data = split(/,/,$_); # assuming data is comma separated

    $sel->wait_for_page_to_load(10000);

    $sel->type('input1',$data[0]);
    $sel->type('input2',$data[1]);
    $sel->type('input3',$data[2]);
    $sel->type('input4',$data[3]);

    $sel->click('submit_button');
}
close $fh;

Selenium is especially useful if the page requires javascript to work properly (a lot of sites do these days). If not you can probably use something simpler like WWW::Mechanize.

Another nice thing about Selenium RC is that it is programming language agnostic. There are lots of libraries for lots of languages that interfaces with Selenium RC.