Python __setattr__ and __getattr__ for global scope?

Question

Suppose I need to create my own small DSL that would use Python to describe a certain data structure. E.g. I'd like to be able to write something like

f(x) = some_stuff(a,b,c)

and have Python, instead of complaining about undeclared identifiers or attempting to invoke the function some_stuff, convert it to a literal expression for my further convenience.

It is possible to get a reasonable approximation to this by creating a class with properly redefined __getattr__ and __setattr__ methods and use it as follows:

e = Expression()
e.f[e.x] = e.some_stuff(e.a, e.b, e.c)

It would be cool though, if it were possible to get rid of the annoying "e." prefixes and maybe even avoid the use of []. So I was wondering, is it possible to somehow temporarily "redefine" global name lookups and assignments? On a related note, maybe there are good packages for easily achieving such "quoting" functionality for Python expressions?

Answer 1

You might want to take a look at the ast or parser modules included with Python to parse, access and transform the abstract syntax tree (or parse tree, respectively) of the input code. As far as I know, the Sage mathematical system, written in Python, has a similar sort of precompiler.

Answer 2

I'm not sure it's a good idea, but I thought I'd give it a try. To summarize:

class PermissiveDict(dict):
    default = None

    def __getitem__(self, item):
        try:
            return dict.__getitem__(self, item)
        except KeyError:
            return self.default

def exec_with_default(code, default=None):
    ns = PermissiveDict()
    ns.default = default
    exec code in ns
    return ns

Answer 3

I am also trying to create a DSL. I'm also contemplating what trick can I do to make the expression expressive and beautiful. To compose the expression as string and put it in exec ... in ... does not feel perfectly right for me.

Right now I have some idea. I'm only at a starting point so far. I could compose the expression using lambda like your second expression:

  f = lambda x: some_stuff(x.a, x.b, x.c)

Those x really feels superfluous here. So I would elide them like

  f = lambda: some_stuff(a, b, c)

This is as clean an expression as I can think of. It passes the Python compiler. But how can I execute them? I still haven't figure out. In my case I may need some ast magic because my function actually have different semantics than Python.

Let me know if you have found some useful technique. I may look the Sage mathematical system too to see if I can learn something.

Answer 4

In response to Wai's comment, here's one fun solution that I've found. First of all, to explain once more what it does, suppose that you have the following code:

definitions = Structure()
definitions.add_definition('f[x]', 'x*2')
definitions.add_definition('f[z]', 'some_function(z)')
definitions.add_definition('g.i', 'some_object[i].method(param=value)')

where adding definitions implies parsing the left hand sides and the right hand sides and doing other ugly stuff. Now one (not necessarily good, but certainly fun) approach here would allow to write the above code as follows:

@my_dsl
def definitions():
    f[x] = x*2
    f[z] = some_function(z)
    g.i  = some_object[i].method(param=value)

and have Python do most of the parsing under the hood. The idea is based on the simple exec <code> in <environment> statement, mentioned by Ian, with one hackish addition. Namely, the bytecode of the function must be slightly tweaked and all local variable access operations (LOAD_FAST) switched to variable access from the environment (LOAD_NAME).

It is easier shown than explained: http://kt.pri.ee/stuff/pydsl/

There are various tricks you may want to do to make it practical. For example, in the code presented at the link above you can't use builtin functions and language constructions like for loops and if statements within a @my_dsl function. You can make those work, however, by adding more behaviour to the Env class.

Update. Here is a slightly more verbose explanation of the same thing.

Answer 5

Regarding Ian's answer. It's interesting that this doesn't work:

def f():
  print hello

exec_with_default(f.func_code, 'world')

Any idea why?