How to configure Java's SaxParserFactory to disable entity checking?

Question

I am writing a screen scraping app that reads out various pages and extracts the data. I'm using the SAXParserFactory go get a SAXParser which in turn gets me an XMLReader. I have configured the Factory like this:

spf = SAXParserFactory.newInstance();
spf.setValidating(false);
spf.setFeature("http://xml.org/sax/features/validation", false);
spf.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
spf.setFeature("http://xml.org/sax/features/use-entity-resolver2", false);

However, whenever I parse a document that contains the &nbsp entity I get an

SEVERE: null
    org.xml.sax.SAXParseException: The
    entity "nbsp" was referenced, butnot declared.
            at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1231)
            at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)

I can understand that it can't find the entity, since I told the factory to not read the DTD, but how do I disable entity checking alltogther?

EDIT: This is for an Android app, which is why I am reluctant to use an API/library that isn't in the standard environment.

Answer 1

I think it is possible to intercept these errors by writing your own DOMErrorHandler instance - more details here: http://xerces.apache.org/xerces2-j/faq-write.html

I've used this approach to work around a problem whereby I'm parsing a drawing as a XML SVG document generated by Corel Draw 12 which breaks the SVG DTD rules sometimes in the documents it outputs.

Why have you told it not to read the DTD? Is that because you don't want it to access this from the W3C servers by connecting to the internet; you want a standalone, off-network solution with a local DTD? I needed the same: I downloaded the SVG DTD and modules locally and used this Java library to force local DTD access: http://doctypechanger.sourceforge.net/

Answer 2

It seems to me that you've disabled the parser's capability to understand what to do with  . What would you expect the SAX parser to do given that it doesn't understand this entity at all.

Perhaps if you're scraping HTML, you may be better off using JTidy ? It's an HTML parser that presents the HTML in a DOM for further analysis.

Answer 3

If it's HTML pages that you're reading, I'd strongly recommend using one of the libraries that deals with the fact that even valid HTML isn't XML and most HTML isn't valid. Try one of these:

• NekoHTML
• TagSoup

Edit: Just saw that it's an Android app. That is going to make it tougher. NekoHTML comes in at 109kb and TagSoup at 89kb.

Answer 4

SAX doesn't seem capable of this, but the StAX API does. See this previous question/answer for how to set this up.

If you're writing the XML processor by hand, the StAX API is a lot easier to deal with than the SAX API, so you win on both counts.