views:

78

answers:

3

pQuery is a pragmatic port of the jQuery JavaScript framework to Perl which can be used for screen scraping.

pQuery quite sensitive to malformed HTML. Consider the following example:

use pQuery;

my $html_malformed = "<html><head><title>foo</title></head><body>bar</body></html>>";
my $page = pQuery($html_malformed);
my $title = $page->find("title");
print "The title is: ", $title->html, "\n";

pQuery won't find the title tag in the example above due to the double ">>" in the malformed HTML.

To make my pQuery based applications more tolerant to malformed HTML I need to pre-process the HTML by cleaning it up before passing it to pQuery.

Starting with the code fragment given above, what is the most robust pure-perl way to clean-up the HTML to make it parse:able by pQuery?

+1  A: 

Try HTML::Tidy, which fixes invalid HTML.

lonesomeday
Sorry, but I need a pure-perl solution. It has now been clarified in the question. Thanks for the answer anyways! :-)
knorv
A: 

is that what you want?

$html_malformed =~ r|<+(<.*?>)>+|$1|g;
elektronikLexikon
No, that would only catch the example given. I'm looking for a more general solution.
knorv
+3  A: 

I'd report this as a bug in pQuery. Here's a workaround:

use HTML::TreeBuilder;
use pQuery;

my $html_malformed = "<html><head><title>foo</title></head><body>bar</body></html>>";
my $html_cleaned = HTML::TreeBuilder->new_from_content($html_malformed);
my $page = pQuery($html_cleaned->as_HTML);
$html_cleaned->delete;
my $title = $page->find("title");
print "The title is: ", $title->html, "\n";

This doesn't make a lot of sense, since pQuery already uses HTML::TreeBuilder as its underlying parsing mechanism, but it does work.

cjm