Python LEPL LineAwareConfiguration trouble

Question

Hello. I'm new to using python's LEPL, and it looks great. However, I'm trying to use the LineAwareConfiguration to easily handle a grammar in which whitespace matters, and I'm running into some serious issues. Namely, this code works:

from lepl import *
broken = Token(r'.')
parser = broken[:].string_parser()

While this code yields a UnicodeEncodeError:

from lepl import *
broken = Token(r'.')
parser = broken[:].string_parser(LineAwareConfiguration())

I've tried debugging on my own, and the code is way too complex for me to understand what is going on. Does anyone know LEPL well enough to tell me why this isn't working? LineAwareConfiguration seems pretty broken in this case, and it's not exactly a corner case.

It should be noted that other tokens (such as Token(r'[^a]')) also throw the unicode error, not just one with r'.'.

Can anyone help?

Answer 1

Never mind: it looks like this is a known bug and that it's being worked on. Shows me to ask a question before I check the mailing lists...

Answer 2

yeah, sorry about this. it's at least partly related to the difference between normal (byte) and unicode strings in python 2.6 (the line aware code is new this last release). i will get a fixed release out as soon as i can - hopefully by the weekend. if you keep watching the mailing list there may be a point before then at which there's something in hg that would be worth testing.

at the moment (ie this morning before work - this is not in hg yet) i have everything compiling, but various tokens now simply fail to match.

believe me, i would have preferred the first lepl related post here to be something more positive than this, so i am working on it.... :)

[later - 2009-11-22] ok, i think that release 3.3.2 fixes this. the problem was that, although in general unicode and "normal" strings inter-operate, an error is generated if a normal string is used as a template to format a unicode string. so now all templates are converted to unicode.