What cool hacks can be done using sys.settrace?

Question

I love being able to modify the arguments the get sent to a function, using settrace, like :

import sys

def trace_func(frame,event,arg):
    value = frame.f_locals["a"]
    if value % 2 == 0:
        value += 1
        frame.f_locals["a"] = value

def f(a):
    print a

if __name__ == "__main__":
    sys.settrace(trace_func)
    for i in range(0,5):
        f(i)

And this will print:

1
1
3
3
5

What other cool stuff can you do using settrace?

Answer 1

I don't have an exhaustively comprehensive answer but one thing I did with it, with the help of another user on SO, was create a program that generates the trace tables of other Python programs.

Answer 2

Of course, code coverage is accomplished with the trace function. One cool thing we haven't had before is branch coverage measurement, and that's coming along nicely, about to be released in an alpha version of coverage.py.

So for example, consider this function:

def foo(x):
    if x:
        y = 10
    return y

if you test it with this call:

assert foo(1) == 10

then statement coverage will tell you that all the lines of the function were executed. But of course, there's a simple problem in that function: calling it with 0 raises a UnboundLocalError.

Branch measurement would tell you that there's a branch in the code that isn't fully exercised, because only one leg of the branch is ever taken.

Answer 3

I would strongly recommend against abusing settrace. I'm assuming you understand this stuff, but others coming along later may not. There are a few reasons:

1. Settrace is a very blunt tool. The OP's example is a simple one, but there's practically no way to extend it for use in a real system.

2. It's mysterious. Anyone coming to look at your code would be completely stumped why it was doing what it was doing.

3. It's slow. Invoking a Python function for every line of Python executed is going to slow down your program by many multiples.

4. It's usually unnecessary. The original example here could have been accomplished in a few other ways (modify the function, wrap the function in a decorator, call it via another function, etc), any of which would have been better than settrace.

5. It's hard to get right. In the original example, if you had not called f directly, but instead called g which called f, your trace function wouldn't have done its job, because you returned None from the trace function, so it's only invoked once and then forgotten.

6. It will keep other tools from working. This program will not be debuggable (because debuggers use settrace), it will not be traceable, it will not be possible to measure its code coverage, etc. Part of this is due to lack of foresight on the part of the Python implementors: they gave us settrace but no gettrace, so it's difficult to have two trace functions that work together.

Trace functions make for cool hacks. It's fun to be able to abuse it, but please don't use it for real stuff. If I sound hectoring, I apologize, but this has been done in real code, and it's a pain. For example, DecoratorTools uses a trace function to perform the magic feat of making this syntax work in Python 2.3:

# Method decorator example
from peak.util.decorators import decorate

class Demo1(object):
    decorate(classmethod)   # equivalent to @classmethod
    def example(cls):
        print "hello from", cls

A neat hack, but unfortunately, it meant that any code that used DecoratorTools wouldn't work with coverage.py (or debuggers, I guess). Not a good tradeoff if you ask me. I changed coverage.py to provide a mode that lets it work with DecoratorTools, but I wish I hadn't had to.

Even code in the standard library sometimes gets this stuff wrong. Pyexpat decided to be different than every other extension module, and invoke the trace function as if it were Python code. Too bad they did a bad job of it.

</rant>