ansaurus

Question

Open source java library for HTML to text conversion

Answer 1

+1 A:

HtmlUnit, it even shows the page after processing JavaScript/Ajax.

Ahmed Ashour 2009-10-05 07:37:12

I see how it gives me the response as HTML, not text

David Rabinowitz 2009-10-05 08:07:47

Check .asText() [http://htmlunit.sourceforge.net/apidocs/com/gargoylesoftware/htmlunit/html/DomNode.html#asText()]

Ahmed Ashour 2009-10-05 08:16:51

Thanks. I went for Jericho at the end, but I'll keep HtmlUnit in mind

David Rabinowitz 2009-10-05 19:13:31

Answer 2

A:

I use TagSoup, it is available for several languages and does a really good job with HTML found "in the wild". It produces either a cleaned up version of the HTML or XML, that you can then process with some DOM/SAX parser.

Rich Seller 2009-10-05 07:57:16

Thanks, but I need the final result in plain text

David Rabinowitz 2009-10-05 08:08:29

Once it is in XML, you can implement a SAX parser to output only the text nodes (e.g. a DefaultHandler no-op implementations of all methods apart from `characters`)

Rich Seller 2009-10-05 08:38:30

Answer 3

+2 A:

Try Jericho.

The TextExtractor class sounds like it will do what you want. Sorry can't post a 2nd link as I'm a new user but scroll down the homepage a bit and there's a link to it.

Chris R 2009-10-05 12:14:16

Here's the link to that class: http://jericho.htmlparser.net/docs/javadoc/net/htmlparser/jericho/TextExtractor.html

Chris R 2009-10-05 12:15:04

Thanks! I actually used the Renderer at the end

David Rabinowitz 2009-10-05 13:40:37

ansaurus

tags:

views:

answers:

Open source java library for HTML to text conversion

related questions