exec() bytecode with arbitrary locals?

Question

Suppose I want to execute code, for example

    value += 5

inside a namespace of my own (so the result is essentially mydict['value'] += 5). There's a function exec(), but I have to pass a string there:

    exec('value += 5', mydict)

and passing statements as strings seems strange (e.g. it's not colorized that way). Can it be done like:

    def block():
        value += 5

    ???(block, mydict)

? The obvious candidate for last line was exec(block.__code__, mydict), but no luck: it raises UnboundLocalError about value. I believe it basically executes block(), not the code inside block, so assignments aren't easy – is that correct?

Of course, another possible solution would be to disassembly block.__code__...

FYI, I got the question because of this thread. Also, this is why some (me undecided) call for new syntax

    using mydict: 
        value += 5

---

Note how this doesn't throw error but doesn't change mydict either:

    def block(value = 0):
        value += 5

    block(**mydict)

Answer 1

You can pass bytecode instead of a string to exec, you just need to make the right bytecode for the purpose:

>>> bytecode = compile('value += 5', '<string>', 'exec')
>>> mydict = {'value': 23}
>>> exec(bytecode, mydict)
>>> mydict['value']
28

Specifically, ...:

>>> import dis
>>> dis.dis(bytecode)
  1           0 LOAD_NAME                0 (value)
              3 LOAD_CONST               0 (5)
              6 INPLACE_ADD         
              7 STORE_NAME               0 (value)
             10 LOAD_CONST               1 (None)
             13 RETURN_VALUE

the load and store instructions must be of the _NAME persuasion, and this compile makes them so, while...:

>>> def f(): value += 5
... 
>>> dis.dis(f.func_code)
  1           0 LOAD_FAST                0 (value)
              3 LOAD_CONST               1 (5)
              6 INPLACE_ADD         
              7 STORE_FAST               0 (value)
             10 LOAD_CONST               0 (None)
             13 RETURN_VALUE

...code in a function is optimized to use the _FAST versions, and those don't work on a dict passed to exec. If you started somehow with a bytecode using the _FAST instructions, you could patch it to use the _NAME kind instead, e.g. with bytecodehacks or some similar approach.

Answer 2

Use the global keyword to force dynamic scoping on any variables you want to modify from within the block:

def block():
    global value
    value += 5

mydict = {"value": 42}
exec(block.__code__, mydict)
print(mydict["value"])

Answer 3

From S.Lott's comment above I think I get the idea for an answer using creation of new class.

class _(__metaclass__ = change(mydict)):
    value += 1
    ...

where change is a metaclass whose __prepare__ reads dictionary and whose __new__ updates dictionary.

For reuse, the snippet below would work, but it's kind of ugly:

def increase_value(d):
    class _(__metaclass__ = change(d)):
        value += 1
        ...

increase_value(mydict)

Answer 4

Here is a crazy decorator to create such a block that uses "custom locals". In reality it is a quick hack to turn all variable access inside the function to global access, and evaluate the result with the custom locals dictionary as environment.

import dis
import functools
import types
import string

def withlocals(func):
    """Decorator for executing a block with custom "local" variables.

    The decorated function takes one argument: its scope dictionary.

    >>> @withlocals
    ... def block():
    ...     counter += 1
    ...     luckynumber = 88

    >>> d = {"counter": 1}
    >>> block(d)
    >>> d["counter"]
    2
    >>> d["luckynumber"]
    88
    """
    def opstr(*opnames):
        return "".join([chr(dis.opmap[N]) for N in opnames])

    translation_table = string.maketrans(
            opstr("LOAD_FAST", "STORE_FAST"),
            opstr("LOAD_GLOBAL", "STORE_GLOBAL"))

    c = func.func_code
    newcode = types.CodeType(c.co_argcount,
                             0, # co_nlocals
                             c.co_stacksize,
                             c.co_flags,
                             c.co_code.translate(translation_table),
                             c.co_consts,
                             c.co_varnames, # co_names, name of global vars
                             (), # co_varnames
                             c.co_filename,
                             c.co_name,
                             c.co_firstlineno,
                             c.co_lnotab)

    @functools.wraps(func)
    def wrapper(mylocals):
        return eval(newcode, mylocals)
    return wrapper

if __name__ == '__main__':
    import doctest
    doctest.testmod()

This is just a monkey-patching adaption of someone's brilliant recipe for a goto decorator