How to get DOMDocument to be nice to ASCII control characters?

views:

211

answers:

How to get DOMDocument to be nice to ASCII control characters?

The HTML document which I am parsing contains some ASCII control codes. I noticed that PHP's DOMDocument parser truncates text nodes when it finds ASCII control characters within the node, such as

Device Control 0x13

End of Medium 0x19

File Separator 0x1C

Group Separator 0x1D

Is this a bug or a feature? Is there any way to have DOMDocument act otherwise? I resorted to remove this characters before DOM processing, but I wonder if that's the right solution.

+2 A:

Probably both a bug and a feature.

XML 1.0 is very restrictive about the ASCII control characters that it will accept. So it seems like your DOMDocument is trying to protect you from yourself by truncating (although it should return some indication of a problem, so I'd call that a bug).

XML 1.1 is less restrictive; the only thing that it doesn't allow is NUL. So, one possible solution is to configure your DOMDocument object so that it knows it should be managing 1.1.

Edit: it looks like you can pass the XML version number to the DOMDocument constructor (but I'm not a PHP programmer, so don't know if I'm reading the docs correctly).

Edit 2: I just reread your question, and realized that your parsing, not constructing. If you prepend a valid 1.1 prologue to the input, that should be a workaround. Or perhaps by constructing the DOMDocument with the correct version number, it will parse correctly without that prologue.

kdgregory 2010-01-04 17:29:48

Even in XML 1.1, the listed characters are RestrictedChar so have to be encoded as character references rather than included as-is!

bobince 2010-01-04 17:51:06

I'm not sure that's correct. I assume that you're looking at the `document` production, which contains `( Char* RestrictedChar Char* )`. However, if you follow the links, you'll see that `Char` and `RestrictedChar` overlap, and there's no mention of requiring `RestrictedChar` to be encoded as an entity (at least that I could find) -- they're just identified as "discouraged."

kdgregory 2010-01-04 18:17:03

That's a `-` in front of `(Char* RestrictedChar Char*)`, which means the document must not have RestrictedChar anywhere in it. Indeed, none of the XML parsers I have here will touch such a document.

bobince 2010-01-04 20:16:01

+1 A:

To add to what kdgregory said about allowable characters, those control characters are not just invalid in XML, but also in SGML. So legacy HTML or XHTML, you're not ever going to be able to parse any of those characters in an HTML document.

(And I have no idea why you would want to. Binary data doesn't belong in [X][HT]ML.)

bobince 2010-01-04 17:50:26

Thanks. Anyway, I don't want to add those characters! I'm parsing a web document that someone else authored. These chars were probably pasted from another application (Microsoft?) directly into the web editor or something of that ilk.

bosh 2010-01-04 18:04:11

ansaurus

tags:

views:

answers:

How to get DOMDocument to be nice to ASCII control characters?

related questions