How to ensure xml.dom.minidom can parse its own output?

Question

I'm trying to serialize some data to xml in a way that can be read back. I'm doing this by manually building a DOM via xml.dom.minidom, and writing it to a file using the included writexml method.

Of particular interest is how I build the text nodes. I do this by initializing a Text object and then setting its data attribute. I'm not sure why the Text object doesn't take its content in the constructor, but that's just the way it in simplemented in xml.dom.minidom.

To give a concrete example, the code looks something like this:

import xml.dom.minidom as dom
e = dom.Element('node')
t = dom.Text()
t.data = "The text content"
e.appendChild(t)
dom.parseString(e.toxml())

This seemed reasonable to me, particularly since createTextNode itself is implemented exactly like this:

def createTextNode(self, data):
    if not isinstance(data, StringTypes):
        raise TypeError, "node contents must be a string"
    t = Text()
    t.data = data
    t.ownerDocument = self
    return t

The problem is that setting the data like this allows us to write text that later cannot be parsed back. To give an example, I am having difficulty with the following character:

you´ll

The quote is ord(180), '\xb4'. My question is, what is the correct procedure to encode this data into an xml document suck that I parse the document with minidom to restore the original tree?

Answer 1

The issue you're encountering, as explained in Python's online docs, is that of Unicode encoding:

Node.toxml([encoding])
Return the XML that the DOM represents as a string.

With no argument, the XML header does not specify an encoding, and the result is
Unicode string if the default encoding cannot represent all characters in the 
document. Encoding this string in an encoding other than UTF-8 is likely
incorrect, since UTF-8 is the default encoding of XML.

With an explicit encoding [1] argument, the result is a byte string in the 
specified encoding. It is recommended that this argument is always specified.
To avoid UnicodeError exceptions in case of unrepresentable text data, the 
encoding argument should be specified as “utf-8”.

So, call .toxml('utf8'), not just .toxml(), and use unicode strings as text contents, and you should be fine for a "round-trip" as you desire. For example:

>>> t.data = u"The text\u0180content"
>>> dom.parseString(e.toxml('utf8')).toxml('utf8')
'<?xml version="1.0" encoding="utf8"?><node>The text\xc6\x80content</node>'
>>>