Why doesn't my WWW::Mechanize submit a PayPal form work?

Question

Nothing I've tried can get my code to submit correctly. Can anyone else figure this out?

#!/usr/bin/perl

use WWW::Mechanize;


my $user = '[email protected]';
my $pass  = 'hackswipe';
# Test account; don't worry


my $browser = WWW::Mechanize->new();
$browser->get("https://www.paypal.com/");
$browser->form_with_fields("login_email", "login_password");
$browser->field("login_email", $user);
$browser->field("login_password", $pass);
$browser->submit_form();
$browser->get("https://www.paypal.com/us/cgi-bin/webscr?cmd=_profile-api-add-direct-access");
##### Help here ---> Trying to submit form with default option selected #####
my $html = $browser->content;

print $html;

Answer 1

We don't know what the problem is, exactly. Have you read the FAQ?

perldoc WWW::Mechanize::FAQ

It gives suggestions on how to debug problems with Mech. The first thing I'd have to ask is if the form is using JavaScript. I'd bet a nickel that PayPal's pages are doing that.

Answer 2

It works for me, but when it comes to debugging web scrapers, etc, you should watch the HTTP transaction. That's really easy to add since WWW::Mechanize is an LWP::UserAgent subclass:

use WWW::Mechanize;

my $browser = WWW::Mechanize->new();

# See LWP::Debug
$browser->add_handler("request_send",  sub { shift->dump; return });
$browser->add_handler("response_done", sub { shift->dump; return });

Now you can see what you send and what PayPal sends back.

Often you can also use various HTTP sniffing tools, but those only work for things that you send in plaintext, so you're out of luck here.

In this case, however, PayPal is on to you. They know you are using a script. Part of the output I get is:

<h2>Request API Credentials</h2>
</div>
<div id="messageBox"></div>
<div id="main"><div class="layout1"><form action="https://www.paypal.com/us/cgi-bin/webscr?dispatch=5885d80a13c0db1f8e263663d3faee8dc18bca4c6f47e633b393e284a5f8a8f8" class="">
<input type="hidden" name="cmd" value="_profile-api-add-direct-access"><input type="hidden" name="api_flow_origin" value=""><input type="hidden" name="show_switch" value="1"><input type="hidden" name="auth_type" value="ssl"><input type="hidden" name="api_username" value=""><input type="hidden" name="program_name" value=""><input type="hidden" name="program_id" value=""><input type="hidden" name="partner_name" value=""><input type="hidden" name="partner_id" value=""><input type="hidden" name="partner_code" value=""><p>API credentials consist of three elements:</p>
<ul>
<li>An API username</li>
<li>An API password</li>
<li>Either an API signature or an API SSL client-side certificate</li>
</ul>
<p>If you’re using a shopping cart or solution provider, ask whether you need an API signature or a certificate.</p>

If you want to interact with PayPal through a program, you need to sign up for developer access.