ansaurus

Question

How can I extract HTML img tags wrapped in anchors in Perl?

Answer 1

A:

Use a proper parser (like HTML::Parser or HTML::TreeBuilder). Using regular expressions to parse SGML (HTML/XML included) isn't really all that effective because of funny multiline tags and attributes like the one you've run into.

fennec 2009-12-30 20:27:16

Answer 2

+3 A:

You should post some examples that you tried with "LinkExtractor, TokenParser, Mechanize & TreeBuilder" so that we can help you.

Here is something which works for me in pQuery:

use pQuery;

my $data = '
  <html>
    <a href="http://www.something.com"&gt;Not yahoo anchor text</a>
    <a href="http://www.yahoo.com"&gt;&lt;img src="somepath/nw.gif" alt="Open this result in new window"></img></a>
    <a href="http://www.yahoo.com"&gt;just text for yahoo</a>
    <a href="http://www.yahoo.com"&gt;anchor text only<img src="blah" alt="alt text"/></a>
  </html>
';

pQuery( $data )->find( 'a' )->each(
    sub {
        say $_->innerHTML 
            if $_->getAttribute( 'href' ) eq 'http://www.yahoo.com';
    }
);

# produces:
#
# => <img alt="Open this result in new window" src="somepath/nw.gif"></img>
# => just text for yahoo
# => anchor text only<img /="/" alt="alt text" src="blah"></img>
#

And if you just want the text:

pQuery( $data )->find( 'a' )->each(
    sub {
        return unless $_->getAttribute( 'href' ) eq 'http://www.yahoo.com';

        if ( my $text = pQuery($_)->text ) { say $text }
    }
);

# produces:
#
# => just text for yahoo
# => anchor text only
#

/I3az/

draegtun 2009-12-30 21:02:28

added a script I tried as an answer.

2009-12-30 22:31:58

When I ran the above pquery script, why do I not see any output?

2009-12-30 22:49:40

Did you get any errors? NB. For `say` u need perl 5.10.* or the Perl6::Say module.

draegtun 2009-12-30 23:16:29

Answer 3

A:

If the HTML you are working with is fairly close to well formed you can usually load it into an XML module that supports HTML and use it to find and extract data from the parts of the document you are interested in. My method of choice is XML::LibXML and XPath.

use XML::LibXML;

my $parser = XML::LibXML->new();
my $html = ...;
my $doc = $parser->parse_html_string($html);

my @links = $doc->findnodes('//a[@href = "http://example.com"]');
for my $node (@links) {
    say $node->textContent();
}

The string passed to findnodes is an XPath expression that looks for all 'a' element descendants of $doc that have an href attribute equal to "http://example.com".

Ven'Tatsu 2009-12-30 21:30:25

Answer 4

A:

I tried this script, but not getting exactly what I want:

use WWW::Mechanize(); my $mech = WWW::Mechanize->new(); my $html = $mech->get('http://umallvt.com/directory.php'); my @links= $mech->find_all_links( text_regex => qr/a/i ); foreach(@links){ if($->url() eq 'http://www.victoriassecret.com/'){ print $->url(); print $_->text(); } } END The out put is : url: http://www.victoriassecret.com/ text: Victoria's Secret

In case the page had an anchor tag like below: a href="http://www.victoriassecret.com/" target=_blank>

The above perl script would give : url: http://www.victoriassecret.com/ text: Open this result in new window But the desired result is: url: http://www.victoriassecret.com/ text: IMAGE

2009-12-30 22:28:59

This is not an answer. Please edit your question instead.

Svante 2009-12-30 23:10:32

Answer 5

A:

I also tried this:

 #!/usr/bin/perl --

 use strict;
 use warnings;
 use HTML::TreeBuilder;

 my $html = <<'__HTML__';
 <a href="http://www.yahoo.com" target=_blank><img
 src="http://us.i1.yimg.com/nw.gif" 
 alt="Open this result in new window">ANCHOR TEXT</a>

 <a href="http://www.yahoo.com" target=_blank><img
 src="http://us.i1.yimg.com/nw.gif" 
 alt="Two clues for the price of one"></a>
 __HTML__


 {
 my $h = HTML::TreeBuilder->new_from_content($html);

 for my $link ( $h->look_down( _tag => q{a}, href => 'http://www.yahoo.com' ) )
      {
      print $link->attr('href'),"\n";
      my $text = $link->as_trimmed_text;
      unless ($text) {
          $text = join ' ',
           map { $_->attr('alt') } $link->look_down( alt => qr/^.+$/ );
           }
     print "$text\n\n";
     } ## end for my $link ( $h->look_down...)

 }

 __END__

The output is :

 http://www.yahoo.com; ANCHOR TEXT

 http://www.yahoo.com; Two clues for the price of one

But the desired output is :

 http://www.yahoo.com; ANCHOR TEXT

 http://www.yahoo.com; IMAGE (indicating no anchor text and also presence of img tag within anchor tag )

Any ideas?

2009-12-30 22:42:42

This is not an answer. Please edit your question instead.

Svante 2009-12-30 23:11:12

ansaurus

tags:

views:

answers:

How can I extract HTML img tags wrapped in anchors in Perl?

related questions