Read XML with multiple top-level items using Python ElementTree?

Question

How can I read an XML file using Python ElementTree, if the XML has multiple top-level items?

I have an XML file that I would like to read using Python ElementTree.

Unfortunately, it has multiple top-level tags. I would wrap <doc>...</doc> around the XML, except I have to put the <doc> after the <?xml> and <!DOCTYPE> fields. But figuring out where <!DOCTYPE> ends is non-trivial.

What I have:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE FOO BAR "foo.dtd" [
<!ENTITY ...>
<!ENTITY ...>
<!ENTITY ...>
]>
<ARTICLE> ... </ARTICLE>
<ARTICLE> ... </ARTICLE>
<ARTICLE> ... </ARTICLE>
<ARTICLE> ... </ARTICLE>

What I want:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE FOO BAR "foo.dtd" [
<!ENTITY ...>
<!ENTITY ...>
<!ENTITY ...>
]>
<DOC>
<ARTICLE> ... </ARTICLE>
<ARTICLE> ... </ARTICLE>
<ARTICLE> ... </ARTICLE>
<ARTICLE> ... </ARTICLE>
</DOC>

NB the name of tag ARTICLE might change, so I cannot grep for it.

Can anyone suggest to me how I can add the enclosing <doc>...</doc> after the XML header, or suggest another workaround?

Answer 1

I wrote the following function to add a toplevel tag after the XML processing instructions. You can now find this code in my common Python library as common.myelementtree.add_toplevel_tag

import re
xmlprocre = re.compile("(\s*<[\?\!])")
def add_toplevel_tag(string):
    """
After all the XML processing instructions, add an enclosing top-level <DOC> tag, and return it.
e.g.
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE FOO BAR "foo.dtd" [ <!ENTITY ...> <!ENTITY ...> <!ENTITY ...> ]> <ARTICLE> ...
</ARTICLE> <ARTICLE> ... </ARTICLE> <ARTICLE> ... </ARTICLE> <ARTICLE> ... </ARTICLE>
=>
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE FOO BAR "foo.dtd" [ <!ENTITY ...> <!ENTITY ...> <!ENTITY ...> ]><DOC> <ARTICLE> ...
</ARTICLE> <ARTICLE> ... </ARTICLE> <ARTICLE> ... </ARTICLE> <ARTICLE> ... </ARTICLE></DOC>
"""
    def _advance_proc(string, idx):
        # If possible, advance over whitespace and one processing
        # instruction starting at string index idx, and return its index.
        # If not possible, return None
        # Find the beginning of the processing instruction
        m = xmlprocre.match(string[idx:])
        if m is None: return None
        #print "Group", m.group(1)
        idx = idx + len(m.group(1))
        #print "Remain", string[idx:]

        # Find closing > bracket
        bracketdebt = 1
        while bracketdebt > 0:
            if string[idx] == "<": bracketdebt += 1
            elif string[idx] == ">": bracketdebt -= 1
            idx += 1
        #print "Remain", string[idx:]
        return idx
    loc = 0
    while 1:
        # Advance one processing instruction
        newloc = _advance_proc(string, loc)
        if newloc is None: break
        else: loc = newloc
    return string[:loc] + "<DOC>" + string[loc:] + "</DOC>"