I'm interested in a parser that could take a malformed HTML page, and turn it into well formed HTML before performing some XPath queries on it. Do you know of any?
+1
A:
Unless you're looking to learn more about wheels, use the HTML Tidy code.
AnonJr
2009-10-27 21:02:12
With the plethora of task-specific parser available to a Perl programmer, that is rarely necessary.
Sinan Ünür
2009-10-27 21:12:00
Its been 5 years since I last worked with Perl... guess its showing.
AnonJr
2009-10-28 11:15:49
+11
A:
You should not use an XML parser to parse HTML. Use an HTML parser.
Note that the following is perfectly valid HTML (and an XML parser would choke on it):
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Is this valid?</title>
</head>
<body>
<p>This is a paragraph
<table>
<tr> <td>cell 1 <td>cell 2
<tr> <td>cell 3 <td>cell 4
</table>
</body>
</html>
There are many task specific (in addition to the general purpose) HTML parsers on CPAN. They have worked perfectly for me on an immense variety of extremely messy (and most of the time invalid) HTML.
It would be possible to give specific recommendations if you can specify the problem you are trying to solve.
There is also HTML::TreeBuilder::XPath which uses HTML::Parser to parse the document into a tree and then allows you to query it using XPath. I have never used it but see Randal Schwartz's HTML Scraping with XPath.
Given the HTML file above, the following short script:
#!/usr/bin/perl
use strict; use warnings;
use HTML::TreeBuilder::XPath;
my $tree= HTML::TreeBuilder::XPath->new;
$tree->parse_file("valid.html");
my @td = $tree->findnodes_as_strings('//td');
print $_, "\n" for @td;
outputs:
C:\Temp> z cell 1 cell 2 cell 3 cell 4
The key point here is that the document was parsed by an HTML parser as an HTML document (despite the fact that we were able to query it using XPath).
Sinan Ünür
2009-10-27 22:16:53
+1
A:
You could rephrase the question like this:
I'm interested in a parser that could take a malformed
HTML pageC source, and turn it into well formedHTMLC source before performing someXPath queriescompilation and linking on it. Do you know of any?
Now the question may be a bit more obvious: it's not going to be easy. If it's truly malformed HTML, you may need to do the work by hand until it can be fed into an HTML parser. Then, you can use any of the other modules presented here to do the work. It's unlikely though that you could ever programatically translate raw HTML into strictly valid xhtml.
Robert P
2009-10-27 23:26:05