How can I handle HTTP redirects in my Perl application?

Question

I use this Perl subroutine to get a line from a webpage, which I then split to get the information I require to proceed. This has worked for a dozen years.

sub capture_line {

   my $page_to_get = $_[0];
   my $host_to_get_text = $_[1];
   my $port = 80;
   my $buf = &HTTPGet($page_to_get, $host_to_get_text, $port);
   my $image_capture_text;
   my @lines = split(/\n/,$buf);
#      print "$lines[1]\n";
#      print "$page_to_get, $host_to_get_text\n";
#      print "$buf\n";

    foreach (@lines) {
       if (/$text_to_find/i) {
          $image_capture_text = $_;
     print "in_loop";
       last;
       }

     }
    return $image_capture_text;
}

Unforntuately, $page_to_get is now always a 301 redirect and $buf, when printed, gives me a 301 redirection page, which obviously does not contain the sought after text. Is there a $in value pair (for example) that I can use with HTTPGet to hop me over the redirection so that I get the page that I see when I type http://$host_to_get_text$page_to_get into my browser? Or is there a better way to accomplish the same thing (knowlege of an ever changing filename in the source of a viewed webpage)?

Thank you for your time. Greg Marsh

Answer 1

Where is the HTTPGet function coming from?

If you were to use LWP (http://search.cpan.org/dist/libwww-perl/) to do the HTTP fetching, that will automatically follow redirects (you can specify how many times you want it to follow a redirect before giving up).

e.g.:

   use LWP::Simple qw()
   my ($page_to_get, $host_to_get_text) = @_;
   my $url = "http://$host_to_get_text$page_to_get";
   my $buf = LWP::Simple::get($url);
   my $image_capture_text;
   my @lines = split(/\n/,$buf);
   # ...