How to programatically execute a CGI script form that contains a "file" type upload input field?

Question

I've just written (in Perl) a simple web service that displays a web form containing a single "file" type upload field and a single Submit button. I made this simple web page with the intent that a human could use it and also that a script on a different machine could also execute it programatically. I got the human part all done and tested and now I just realized that I have no idea how to construct a URL on the second machine that successfully executes this cgi script on the first machine (and gets a file uploaded from the second machine to the first machine).

I do have some experience constructing URLs that contain form fields, but have never used the "file" type upload input field before, so I'm struggling with it a bit. Maybe I'm using it wrong.

If my simple web service is a cgi-bin script called "myscript.pl" and if the file upload field is called "uploadthis" and if the file on the second machine that I want to upload programatically is located in "C:\myfile.txt", then shouldn't I be able to just construct a URL in a script on the second machine of the form "http://machine1/cgi-bin/myscript.pl?uploadthis=C:\myfile.txt" and that file gets uploaded programatically just as if a human had clicked on the browse button on the web form manually and chose that file and clicked on the Submit button?

It works fine manually when I bring up the cgi script form on the first machine and browse to the file and click Submit. The file seems to get uploaded by the web browser just fine. But when I type in the above URL into my browser to test out the file upload URL manually before I program that URL into a script on another machine, the file that gets uploaded is a zero length file. It creates a file but it's completely empty. Same results if I use a known bad filename like "http://machine1/cgi-bin/myscript.pl?uploadthis=C:\myfilethatdoesntexist.txt". What am I doing wrong?

I'm trying to accomplish this task on a couple of Windows XP PCs with Apache 2.2 and ActivePerl 5.10.0 and the form method is "post".

Answer 1

First of all, post parameters do not end up at the end of the URL. In addition, it is not possible to send a file upload completely automated from a browser (not even with JavaScript), just to prevent users from drive-by uploads.

If you want to upload from a browser, you will always have to select the file or type the filename into the box, or click the submit button manually. If all of these is automated, the file will not be uploaded.

To test file uploads programmatically, you can use curl:

curl -F "uploadthis=@C:\myfile.txt" http://machine1/cgi-bin/myscript.pl

You can of course write a program that posts files as multipart/form-data as well.

Answer 2

You should use such modules as LWP and HTTP::Request::Common or WWW::Mechanize. Both has examples with file uploading. WWW::Mechanize is simpler, but you should install it from ppm only. LWP is included into ActiveState.

Answer 3

The phrase "multipart/form-data" was something that I had missed completely. When I Google'd that phrase, I found my answer.

Something similar to this example below is how I solved my problem from a perl script:

use HTTP::Request::Common;
use LWP::UserAgent;

$ua = LWP::UserAgent->new;

my $response = $ua->request(
    POST 'http://machine1/cgi-bin/myscript.pl',
    Content_Type => 'form-data',
    Content      => [ 
                        uploadthis => ["C:\myfile.txt"],
                    ]
    );

if ($response->is_success) {
    print "Success.";
}
else {
    print "Failure.";
}