How to avoid a SAXParseException using Xerces when HTML file includes mdash?

Question

I am using the Xerces implementation within JDK6 to perform XPath queries on an HTML 4.0 transitional document. With the following code:

XPath newXPath = XPathFactory.newInstance().newXPath();
XPathExpression xpathExpr = newXPath.compile(expression);
Object xPathResult = xpathExpr.evaluate(inputSource, XPathConstants.NODESET);

Where inputSource is built from a FileInputStream, I receive the exception:

Caused by: org.xml.sax.SAXParseException: The entity "mdash" was referenced, but not declared.  
 at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:239) 
 at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:283)
 at com.sun.org.apache.xpath.internal.jaxp.XPathExpressionImpl.evaluate(XPathExpressionImpl.java:291)  

This message is also printed to the standard output:

[Fatal Error] :20:43: The entity "mdash" was referenced, but not declared.

How can I avoid this exception?

The HTML file is created from an XSLT transformation from XML. I don't think I necessarily need it to be an —, I'm not sure. The HTML is to be displayed in a Java Swing application.

It's difficult for me to judge what information from my specific implementation is relevant for this problem. Please let me know by comments if more information is needed.

---

So, I was under the bad misconception that HTML was XML (a result of not actually thinking of that at all).

So, given an HTML file, how do I go about solving this problem?

• Giving the parser the DTD for HTML 4?
• Replace — with the equivalent. The HTML is created from an XSLT transform, can the stylesheet be set to replace mdash with the equivalent numeric symbol?
• Is there any libraries which would fix the HTML before it's given to the parser? I've noticed JTidy being mentioned for similar purposes.

Answer 1

Given that HTML is not XML I think you might find lots of problems with trying to parse HTML Transitional with an XML parser. But in case your HTML is actually well-formed XML, the mdash and other entities are usually defined in the DTD. Make sure your parser has the DTD for the document and it should be ok.

Answer 2

The problem is that if the document as presented to Xerces does not have a DTD with mdash declared it is not a well-formed XML document - all entities have to be declared. HTML has a set of "builtin" entities that HTML processors need to know about and these should be in a DTD.

The simplest workaround without a DTD will be to replace mdash by its numeric equivalent (— or —)

Answer 3

The right DTD in the header of your file should contain all the necessary entities declarations, and if your file is well-formed then parser will honor this information.

If there's a chance that HTML is not well-formed, I'd recommend TagSoup library for fixing this. It reads input and try to produce valid XHTML as output, never reporting any parse errors, just trying to fix them using built-in heuristics. I was able to successfully process very broken HTML from the web and perfrom XPath queries over it (seems like this is what you need).

Answer 4

I think I found the problem for my specific situation. The HTML file was generated from an XML file using XSLT. By changing the line:

<xsl:output method="html" />

to:

<xsl:output method="xml" />

The transformation did not create the — entity. The output file could then be parsed using Xerces.

I'm not sure if this is "correct", but it seems to do the trick for displaying in Swing.