How can I download a file using WWW::Mechanize or any Perl module?

Question

Is there a way in WWW::Mechanize or any Perl module to read on a file after accessing a website. For example, I clicked a button 'Receive', and a file (.txt) will appear containing a message. How will I be able to read the content? Answers are very much appreciated.. I've been working on this for days,, Also, I tried all the possibilities. Can anyone help? If you can give me an idea please? :)

Here is a part of my code:

...

my $username = "admin";<br>
my $password = "12345";<br>

my $url = "http://...do_gsm_sms.cgi";

my $mech = WWW::Mechanize->new(autocheck => 1, quiet => 0, agent_alias =>$login_agent, cookie_jar => $cookie_jar);

$mech->credentials($username, $password);<br>
$mech->get($url);

$mech->success() or die "Can't fetch the Requested page";<br>

print "OK! \n"; #This works <br> 

$mech->form_number(1);

$mech->click()

;

After this, 'Downloads' dialog box will appear so I can save the file (but I can also set the default to open it immediately instead of saving). Question is, how can I read the content of this file?

..

Answer 1

After the click (assuming that's doing what it's supposed to), the returned data should be stored in your $mech object. You should be able to get the file data with $mech->content(), perhaps after verifying success with $mech->status() and the type of response with $mech->content_type().

You may find it helpful to remember that WWW::Mechanize replaces the browser; anything a browser would have done, like bringing up a download window and saving a file, doesn't actually happen, but all the information the browser would have had is accessible through WWW::Mechanize's methods.

Answer 2

Dare I ask... have you tried this?

my $content = $mech->content();

Answer 3

I take you mean that the web site responds to the form submission by returning a non-HTML response (say a 'text/plain' file), that you wish to save.

I believe you want $mech->save_content( $filename )

Added:

First you need to submit the WWW:Mech's form submission, before saving the resulting (text) file. The click is for clicking a button, whereas you want to submit a form, using $mech->submit() or $mech->submit_form( ... ).

#!/usr/bin/perl

use strict;
use warnings;

use WWW::Mechanize;

my $username = "admin";
my $password = "12345";
my $login_agent = 'WWW::Mechanize login-agent';
my $cookie_jar;

#my $url = "http://localhost/cgi-bin/form_mech.pl";
my $url = "http://localhost/form_mech.html";

my $mech = WWW::Mechanize->new(autocheck => 1, quiet => 0, 
               agent_alias => $login_agent, cookie_jar => $cookie_jar
           );

$mech->credentials($username, $password);
$mech->get($url);

$mech->success() or die "Can't fetch the Requested page";

print "OK! \n"; #This works

$mech->submit_form(
   form_number => 1,
);
die "Submit failed" unless $mech->success;

$mech->save_content('out.txt');

Answer 4

Open the file (not 'Downloads' window) as if you were viewing it within your browser; you can save it later with a few lines of code.

Provided you have HTML::TreeBuilder installed:

my $textFile = $mech->content(format => "text");

should get you the text of the resulting window that opens.

Then open a filehandle to write your results in:

open my $fileHandle, ">", "results.txt";
print $fileHandle $textFile;
close $fileHandle;

Answer 5

I do this all the time with LWP, but I'm sure it's equally possible with Mech

I think where you might be going wrong is using Mech to request the page that has the button on it when you actually want to request the content from the page that the button causes to be sent to the browser when clicked.

What you need to do is review the html source of the page with the button that initiates the download and see what the Action associated with the button is. Most likely it will be a POST with some hidden fields or a URL to do a GET.

The Target URL of the Click has the stuff you actually want to get, not the URL of the page with the button on it.

Answer 6

For problems like this, you often have to investigate the complete chain of events that the browser handles. Use an HTTP sniffer tool to see everything the browser is doing until it gets to the file file. You then have to do the same thing in Mech.