How do you get Python to write down the code of a function it has in memory?

Question

When I pass the options in the program (a computational biology experiment) I usually pass them through a .py file.
So I have this .py file that reads like:

starting_length=9
starting_cell_size=1000
LengthofExperiments=5000000

Then I execute the file and get the data. Since the program is all on my machine and no one else has access to it, it is secure in a trivial way.
I can also write a similar file very easily:

def writeoptions(directory):
    options=""
    options+="starting_length=%s%s"%(starting_length,os.linesep)
    options+="starting_cell_size=%s%s"%(starting_cell_size,os.linesep)
    options+="LengthofExperiments=%s%s"%(LengthofExperiments,os.linesep)
...
    open("%s%soptions.py"%(directory,os.sep),'w').write(options)

I want to pass a function as one of the parameters:

starting_length=9
starting_cell_size=1000
LengthofExperiments=5000000

def pippo(a,b):
    return a+b
functionoperator=pippo

And of course in the real experiment the funtion pippo will be much more complex. And different from experiment to experiment.

But what I am unable to do is to write the function automatically. In short I don't know how to generalise the writeoptions function to keep on writing the options, if one of the options is a function. I could of course copy the original file, but this is inelegant, inefficient (because it contains a lot of extra options that are not being used), and generally does not solve the question.

How do you get python to write down the code of a function, as it writes down the value of a variable?

Answer 1

vinko@mithril$ more a.py

def foo(a):
  print a

vinko@mithril$ more b.py

import a
import inspect

a.foo(89)
print inspect.getsource(a.foo)

vinko@mithril$ python b.py
89
def foo(a):
  print a

Answer 2

Are you asking about this?

def writeoptions(directory):
    options=""
    options+="starting_length=%s%s"%(starting_length,os.linesep)
    options+="starting_cell_size=%s%s"%(starting_cell_size,os.linesep)
    options+="LengthofExperiments=%s%s"%(LengthofExperiments,os.linesep)
    options+="def pippo(a,b):%s" % ( os.linesep, )
    options+="    '''Some version of pippo'''%s" % ( os.linesep, )
    options+="    return 2*a+b%s" % ( os.linesep, )
    open("%s%soptions.py"%(directory,os.sep),'w').write(options)

Or something else?

Answer 3

While it is possible to do what you ask (as Vinko has shown), I'd say it is cleaner to share code. Put pippo and his buddies in a submodule that both programs can access.

Answer 4

Instead of diving into the subject of disassemblers and bytecodes (e.g inspect), why don't you just save the generated Python source in a module (file.py), and later, import it?

I would suggest looking into a more standard way of handling what you call options. For example, you can use the JSON module and save or restore your data. Or look into the marshal and pickle modules.

Answer 5

You might also consider some other means of data persistence. In my own (astronomy) research, I've been experimenting with two different means of storing scripts for reproducibility. The first is to have them exclusively inside a subversion repository, and then have the job submission script automatically commit them. For instance, if you just wanted to do this in bash:

alias run_py='svn ci -m "Commit before running"; python2.5 $*'

and inside the script, have the output prefixed by the current subversion revision number for that file, you'd have a record of each script that was run and what the input was. You could pull this back out of subversion as need be.

Another, substantially less full-featured, means of tracking the input to a function could be via something like LodgeIt, a pastebin that accepts XML-RPC input and comes with Python bindings. (It can be installed locally, and has support for replying to and updating existing pastes.)

But, if you are looking for a relatively small amount of code to be included, Vinko's solution using inspect should work quite well. Doug Hellman covered the inspect module in his Python Module of the Week series. You could create a decorator that examines each option and argument and then prints it out as appropriate (I'll use inspect.getargspec to get the names of the arguments.)

import inspect
from functools import wraps

def option_printer(func):
    @wraps(func)
    def run_func(*args, **kwargs):
        for name, arg in zip(inspect.getargspec(func)[0], args) \
                       + sorted(kwargs.items()):
            if isinstance(arg, types.FunctionType): 
                print "Function argument '%s' named '%s':\n" % (name, func.func_name)
                print inspect.getsource(func)
            else:
                print "%s: %s" % (name, arg)
        return func(*args, **kwargs)
    return run_func

This could probably be made a bit more elegant, but in my tests it works for simple sets of arguments and variables. Additionally, it might have some trouble with lambdas.