Python generating Python

Question

Ok, I have a group of objects which I am creating a class for that I want to store each object as its own text file. I would really like to store it as a Python class definition which subclasses the main class I am creating. So, I did some poking around and found a Python Code Generator on effbot.org. I did some experimenting with it and here's what I came up with:

#
# a Python code generator backend
#
# fredrik lundh, march 1998
#
# [email protected]
# http://www.pythonware.com
#
# Code taken from http://effbot.org/zone/python-code-generator.htm

import sys, string

class CodeGeneratorBackend:

    def begin(self, tab="\t"):
        self.code = []
        self.tab = tab
        self.level = 0

    def end(self):
        return string.join(self.code, "")

    def write(self, string):
        self.code.append(self.tab * self.level + string)

    def indent(self):
        self.level = self.level + 1

    def dedent(self):
        if self.level == 0:
            raise SyntaxError, "internal error in code generator"
        self.level = self.level - 1

class Point():
    """Defines a Point. Has x and y."""
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def dump_self(self, filename):
        self.c = CodeGeneratorBackend()
        self.c.begin(tab="    ")
        self.c.write("class {0}{1}Point()\n".format(self.x,self.y))
        self.c.indent()
        self.c.write('"""Defines a Point. Has x and y"""\n')
        self.c.write('def __init__(self, x={0}, y={1}):\n'.format(self.x, self.y))
        self.c.indent()
        self.c.write('self.x = {0}\n'.format(self.x))
        self.c.write('self.y = {0}\n'.format(self.y))
        self.c.dedent()
        self.c.dedent()
        f = open(filename,'w')
        f.write(self.c.end())
        f.close()

if __name__ == "__main__":
    p = Point(3,4)
    p.dump_self('demo.py')

That feels really ugly, is there a cleaner/better/more pythonic way to do this? Please note, this is not the class I actually intend to do this with, this is a small class I can easily mock up in not too many lines. Also, the subclasses don't need to have the generating function in them, if I need that again, I can just call the code generator from the superclass.

Answer 1

This is pretty much the best way to generate Python source code. However, you can also generate Python executable code at runtime using the ast library. (I linked to the Python 3 version because it's more capable than the 2.x version.) You can build code using the abstract syntax tree, then pass it to compile() to compile it into executable code. Then you can use eval to run the code.

I'm not sure whether there is a convenient way to save the compiled code for use later though (ie. in a .pyc file).

Answer 2

We use Jinja2 to fill in a template. It's much simpler.

The template looks a lot like Python code with a few {{something}} replacements in it.

Answer 3

From what I understand you are trying to do, I would consider using reflection to dynamically examine a class at runtime and generate output based on that. There is a good tutorial on reflection (A.K.A. introspection) at diveintopython.org.

You can use the dir() function to get a list of names of the attributes of a given object. The doc string of an object is accessible via the __doc__ attribute. That is, if you want to look at the doc string of a function or class you can do the following:

>>> def foo():
...    """A doc string comment."""
...    pass
...
>>> print foo.__doc__
A doc string comment.

Answer 4

I'm not sure whether this is especially Pythonic, but you could use operator overloading:

class CodeGenerator:
    def __init__(self, indentation='\t'):
        self.indentation = indentation
        self.level = 0
        self.code = ''

    def indent(self):
        self.level += 1

    def dedent(self):
        if self.level > 0:
            self.level -= 1

    def __add__(self, value):
        temp = CodeGenerator(indentation=self.indentation)
        temp.level = self.level
        temp.code = str(self) + ''.join([self.indentation for i in range(0, self.level)]) + str(value)
        return temp

    def __str__(self):
        return str(self.code)

a = CodeGenerator()
a += 'for a in range(1, 3):\n'
a.indent()
a += 'for b in range(4, 6):\n'
a.indent()
a += 'print(a * b)\n'
a.dedent()
a += '# pointless comment\n'
print(a)

This is, of course, far more expensive to implement than your example, and I would be wary of too much meta-programming, but it was a fun exercise. You can extend or use this as you see fit; how about:

• adding a write method and redirecting stdout to an object of this to print straight to a script file
• inheriting from it to customise output
• adding attribute getters and setters

Would be great to hear about whatever you go with :)

Answer 5

Just read your comment to wintermute - ie:

What I have is a bunch of planets that I want to store each as their own text files. I'm not particularly attached to storing them as python source code, but I am attached to making them human-readable.

If that's the case, then it seems like you shouldn't need subclasses but should be able to use the same class and distinguish the planets via data alone. And in that case, why not just write the data to files and, when you need the planet objects in your program, read in the data to initialize the objects?

If you needed to do stuff like overriding methods, I could see writing out code - but shouldn't you just be able to have the same methods for all planets, just using different variables?

The advantage of just writing out the data (it can include label type info for readability that you'd skip when you read it in) is that non-Python programmers won't get distracted when reading them, you could use the same files with some other language if necessary, etc.