views:

409

answers:

2

I'm using Python 2.x [not negotiable] to read XML documents [created by others] that allow the content of many elements to contain characters that are not valid XML characters by escaping them using the _xHHHH_ convention e.g. ASCII BEL aka U+0007 is represented by the 7-character sequence u"_x0007_". Neither the functionality that allows representation of any old character in the document nor the manner of escaping is negotiable. I'm parsing the documents using cElementTree or lxml [semi-negotiable].

Here is my best attempt at unescapeing the parser output as efficiently as possible:

import re
def unescape(s,
    subber=re.compile(r'_x[0-9A-Fa-f]{4,4}_').sub,
    repl=lambda mobj: unichr(int(mobj.group(0)[2:6], 16)),
    ):
    if "_" in s:
         return subber(repl, s)
    return s

The above is biassed by observing a very low frequency of "_" in typical text and a better-than-doubling of speed by avoiding the regex apparatus where possible.

The question: Any better ideas out there?

+1  A: 

You might as well check for '_x' rather than just _, that won't matter much but surely the two-character sequence's even rarer than the single underscore. Apart from such details, you do seem to be making the best of a bad situation!

Alex Martelli
Checking for '_x' is slightly slower (Python 2.6) and doesn't work on Pythons earlier than 2.3.
John Machin
As for Python 2.2 and earlier, you're right @john -- I was kind of assuming `x >= 3` (is ANYbody still stuck with Python 2.2....?! If so, I'm DEEPLY sorry...!!!). As for relative speed, it depends on how many isolated `'_'` you get and how much slowed checking with a regex can be (test to be effing fast in my experience, but the original poster says otherwise) -- URL to any specific benchmark pls?
Alex Martelli
A: 
xml.sax.saxutils.unescape() 

(maybe with htmlentitydefs module)

abafei
John Machin