How to handle/parse ill-formed html into DOM in java?

Question

I am using NekoHtml. It failed parsing html from sites like mercurynews.com into DOM. Any solution to the problem?

Answer 1

Beyond switching to other parsers? If the site has consistent error patterns, you could hot fix them via series of regex before passing on to the parser.

Answer 2

Have you considered Tag Soup?

http://home.ccil.org/~cowan/XML/tagsoup/

Answer 3

You may consider using the Swing HTML parser.

http://www.rkcole.com/articles/swing/HTMLParser.html

Answer 4

I have used the Cobra renderer from the "Lobo Project" (http://lobobrowser.org/cobra.jsp) for parsing less-than-friendly HTML and it has worked well. It's API is also very easy to use.

Hope this helps.

Answer 5

Use JTidy to tidy it before parsing, or better yet use it as the parser

Answer 6

I don't know what "sites like" means, but MercuryNews.com and most news sites have an RSS interface.