Proper usage of JTidy to purify HTML

Question

Hello, I am trying to use JTidy (jtidy-r938.jar) to sanitize an input HTML string, but I seem to have problems getting the default settings right. Often strings such as "hello world" end up as "helloworld" after tidying. I wanted to show what I'm doing here, and any pointers would be really appreciated:

Assume that rawHtml is the String containing the input (real world) HTML. This is what I'm doing:

        Tidy tidy = new Tidy();
        tidy.setPrintBodyOnly(true);

        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        PrintStream ps = new PrintStream(baos);

        tidy.parse(new StringReader(rawHtml), ps);
        return baos.toString("UTF8");   

First off, does anything look fundamentally wrong with the above code? I seem to be getting weird results with this.

For example, consider the following input:

<p class="MsoNormal" style="text-autospace:none;"><font color="black"><span style="color:black;">???</span></font><b><font color="#7f0055"><span style="color:#7f0055;font-weight:bold;">private</span></font></b><font color="black"><span style="color:black;"> String parseDescription</span></font><font>

The output is:

<p class="MsoNormal" style="text-autospace:none;"><font color= "black"><span style="color:black;">&nbsp;&nbsp;&nbsp;</span></font> <b><font color="#7F0055"><span style= "color:#7f0055;font-weight:bold;">private</span></font></b><font color="black"><span style="color:black;">String parseDescription</span></font></p>

So,

"public String parseDescription" becomes "publicString parseDescription"

Thanks in advance!

Answer 1

Here is how we are calling JTidy from Ant. You may infer the API call from it:

<tidy destdir="${build.dir.result}">
  <fileset dir="${src}" includes="**/*.htm"/>
  <parameter name="tidy-mark" value="false"/>
  <parameter name="output-xml" value="no"/>
  <parameter name="numeric-entities" value="yes"/>
  <parameter name="indent-spaces" value="2"/>
  <parameter name="indent-attributes" value="no"/>
  <parameter name="markup" value="yes"/>
  <parameter name="wrap" value="2000"/>
  <parameter name="uppercase-tags" value="no"/>
  <parameter name="uppercase-attributes" value="no"/>
  <parameter name="quiet" value="no"/>
  <parameter name="clean" value="yes"/>
  <parameter name="show-warnings" value="yes"/>
  <parameter name="break-before-br" value="yes"/>
  <parameter name="hide-comments" value="yes"/>
  <parameter name="char-encoding" value="latin1"/>
  <parameter name="output-html" value="yes"/>
</tidy>

Answer 2

Have a look at how JTidy is configured:

StringWriter writer = new StringWriter();
tidy.getConfiguration().printConfigOptions(writer, true);
System.out.println(writer.toString());

Maybe it then get clear what causes the problem.

What is weird? Little example, of actual output and expected... maybe ?

Answer 3

Well, this seems to be a bug in Jtidy. For the exact file which causes problems, refer here:

http://sourceforge.net/tracker/?func=detail&aid=2985849&group_id=13153&atid=113153

Thanks for all the help folks!