Is there any way to programmatically generate Python bytecode?

Question

I want to hack around with the Python interpreter and try creating a small DSL . Is there any module where I can do something like this theoretical code (similar to LINQ expression trees)?

expression_tree = Function(
    Print(
        String('Hello world!')
    )
 )
compile_to_bytecode(expression_tree)

Or would it just be easier to just generate Python source code? Could this be made easier by using C or SWIG or Cython?

Answer 1

Check out the disassembler module found here:

http://docs.python.org/library/dis.html

Answer 2

In Python 2.X, you would typically approach this with the compiler module and its ast sub-module. In Python 3.X, you would use just ast.

Both offer a compile() function that will go from source/AST to "a code object that can be executed by the exec statement or eval()."

Answer 3

Fernando Meyer recently wrote a blog post explaining how to use the # coding directive to specify your own extensions to Python. Example (the actual format definition is in pyspec.py and tokenizer.py):

# coding: pyspec

class Bow:
    def shot(self):
        print "got shot"

    def score(self):
        return 5

describe Bowling:
    it "should score 0 for gutter game":
        bowling = Bow()
        bowling.shot()
        assert that bowling.score.should_be(5)

Answer 4

It's easier to generate Python code and run it. If you do that you can also debug it more easily since there is actual source for the debugger to display. See also Malte Borchs article in the July issue of Python magazine, where he talks about this amongst other things.

Answer 5

Working via ast and compiling the tree into bytecode, as another answer suggests, is probably simplest; generating sources and compiling those, almost as good.

However, to explore lower-level approaches, check out the links from this page; I've found byteplay especially useful (currently doesn't work on 2.6 nor 3.*, only 2.4 or 2.5, but I think fixing it for 2.6 should be easy, as currently discussed in its tracker). I have not used Phil Eby's similarly-featured BytecodeAssembler, but given the author's reputation I'm sure it's worth checking it out!