How do I get WWW::Mechanize to properly handle 302 redirects with URI fragments?

Question

I have a webpage which redirects to another url in the form of http://www.example.com/url.html#midpage.

I'm wondering if there's anyway for WWW::Mechanize to follow http://www.example.com/url.html instead of http://www.example.com/url.html#midpage?

Answer 1

WWW::Mechanize subclasses LWP::UserAgent, so you can still use any of LWP::UserAgent's methods. Thus you can use the simple_request() method, which doesn't automatically handle redirects. It just returns you the response as an HTTP::Resonse object. Which means you can use the is_redirect() and header() methods to get the redirect URI. Which means you can then use the URI module to pull off everything after the #.

Whew!

Your code would look something like this:

my $response = $mech->simple_request( HTTP::Request->new(GET => 'http://www.example.com/') );
if( $response->is_redirect ) {
  my $location = $response->header( "Location" );
  my $uri = new URI( $location );
  my $new_url = $uri->scheme . $uri->opaque;
# And here is where you do the load of the new URL.
}

There may be some twiddling to do, potentially around the header() line, but this would be the general idea.

Answer 2

WWW::Mechanize is a subclass of LWP::UserAgent, so the answer is the same.

If you want to handle the redirect yourself to rewrite URLs, you might want to use a response_done or response_redirect handler. See the "Handlers" section of the LWP::UserAgent docs.

As for "properly", the HTTP specification doesn't say what a client should do with a fragment except in 14.6 in the case of a referrer header (and that's the only place the word "fragment" even shows up).