How do I use Perl's Google::Search to look for specific URLs?

Question

The Google::Search module, which is based on the AJAX Search API, doesn't seems to work very well, or it is just me?

For example, I use firefox to search google for: http://bloggingheads.tv/forum/member.php?u=12129

It brings results.

But when I use the module this way:

$google_search = Google::Search->Web ( q => "http://bloggingheads.tv/forum/member.php?u=12129" );
@result =  $google_search->all;

I get nothing in the array.

Any idea?

Seems like this API doesn't bring the same results like when searching manually, am I missing something?

Answer 1

Looking at the POD for Google::Search, it looks like it expects you to pass search terms to Web, instead of a URL. I downloaded a test script from CPAN, ran it, and it seems to produce expected results:

use strict;
use warnings;
use Google::Search;

my $search = Google::Search->Web(q => "rock");
my $result = $search->first;
while ($result) {
    print $result->number, " ", $result->uri, "\n";
    $result = $result->next;
}
print $search->error->reason, "\n" if $search->error;

__END__

0 http://www.rock.com/
1 http://en.wikipedia.org/wiki/Rock_music
2 http://en.wikipedia.org/wiki/Rock_(geology)
3 http://rockyourphone.com/
4 http://rockhall.com/
5 http://www.co.rock.mn.us/
6 http://www.co.rock.wi.us/
7 http://www.rockride.org/
etc...

I realize this does not specifically answer your question, but perhaps it steers you in the right direction.

Answer 2

I had a similar problem with cyrillic queries. Both Google::Search and REST::Google from CPAN didn't work for me - they were giving back fewer or no results compared to manual test.

Eventually I wrote a scraping module using WWW::Mechanize and HTML::TreeBuilder.

Here's a sample to get result stats:

my $tree = HTML::TreeBuilder->new_from_content($content);

if (my $div = $tree->look_down(_tag => 'div', id => 'resultStats')) {
    my $stats = $div->as_text();
}
else { warn "no stats" }