Trace Table for Python Programs

Question

Is there a way to get the trace table for a Python program? Or for a program to run another program and get its trace table? I'm a teacher trying to flawlessly verify the answers to the tracing problems that we use on our tests.

So, for example, assuming I have a Python program named problem1.py with the following content:

problem1.py

 a = 1
 b = 2

 a = a + b

Executing the presumed program traceTable.py should go as:

 $ python traceTable.py problem1.py
 L || a | b
 1 || 1 |
 2 || 1 | 2
 4 || 3 | 2

(Or the same information with a different syntax)

I've looked into the trace module, and I can't see a way that it supports this.

---

Updated

Ladies and gentlemen: using Ned Batchelder's excellent advice, I give you traceTable.py!

Well.. almost. As you can see in Ned Batchelder's example, frame.f_lineno doesn't always behave intuitively (e.g. both lines 3 & 4 are counted as line 4), but the line numbers are close enough for a fairly good reference. Also, all calculations are correct.

I have tested this with a long program containing an if statement and it gave the correct table (sans the line numbers).

You will also notice that my program is significantly longer than Ned Batchelder's proof of concept due to accounting for the "more interesting ecosystems of data" in larger programs he mentioned. In the scope of using execfile and all the variables needed to manage it and reduce noise (ala ignored_variables) as well as produce proper string output, a lot more code is needed:

traceTable.py

 '''
 Usage: python traceTable.py program

     -program  Python program to be traced
 '''

 import sys

 if len(sys.argv) < 2:
      print __doc__
      exit()
 else:
      file_name = sys.argv[1]

 past_locals = {}
 variable_list = []
 table_content = ""

 ignored_variables = set([
      'file_name',
      'trace',
      'sys',
      'past_locals',
      'variable_list',
      'table_content',
      'getattr',
      'name',
      'self',
      'object',
      'consumed',
      'data',
      'ignored_variables'])

 def trace(frame, event, arg_unused):
      global past_locals, variable_list, table_content, ignored_variables
      relevant_locals = {}
      all_locals = frame.f_locals.copy()
      for k,v in all_locals.items():
           if not k.startswith("__") and k not in ignored_variables:
                relevant_locals[k] = v
      if len(relevant_locals) > 0 and past_locals != relevant_locals:
           for i in relevant_locals:
                if i not in past_locals:
                     variable_list.append(i)
           table_content += str(frame.f_lineno) + " || "
           for variable in variable_list:
                table_content += str(relevant_locals[variable]) + " | "
           table_content = table_content[:-2]
           table_content += '\n'
           past_locals = relevant_locals
      return trace

 sys.settrace(trace)

 execfile(file_name)

 table_header = "L || "
 for variable in variable_list:
      table_header += variable + ' | '
 table_header = table_header[:-2]
 print table_header
 print table_content

When called, it produces the output

 $ python traceTable.py problem1.py
 L || a | b
 2 || 1
 4 || 1 | 2
 4 || 3 | 2

Answer 1

This isn't a use case that the current Python tracing tool support, but it should be possible to build. I don't know how you decide what columns to output. In your sample, a and b are the only local variables, but larger programs would have more interesting ecosystems of data.

Updated: here's a simple proof of concept:

import sys

def trace(frame, event, arg_unused):
    print event, frame.f_lineno, frame.f_locals
    return trace

sys.settrace(trace)

def foo():
    a = 1
    b = 2

    a = a + b

foo()

when run, the output is:

call 9 {}
line 10 {}
line 11 {'a': 1}
line 13 {'a': 1, 'b': 2}
return 13 {'a': 3, 'b': 2}

Answer 2

You could use the Python debugger, though I do not know how to have it step through on it's own, but it's likely do-able, then you could just parse the output.

Here's a really crude example:

adding.py

a = 1
b = 2

a = a + b

running it...

PS >python -m pdb adding.py
> adding.py(1)<module>()
-> a = 1
(Pdb) alias stepprint step;;print a;;print b
(Pdb) stepprint
> adding.py(2)<module>()
-> b = 2
1
*** NameError: name 'b' is not defined
(Pdb) stepprint
> adding.py(4)<module>()
-> a = a + b
1
2
(Pdb) stepprint
--Return--
> adding.py(4)<module>()->None
-> a = a + b
3
2
(Pdb) stepprint
--Return--
> <string>(1)<module>()->None
3
2
(Pdb) stepprint
The program finished and will be restarted
> adding.py(1)<module>()
-> a = 1
*** NameError: name 'a' is not defined
*** NameError: name 'b' is not defined
(Pdb) q

PS >

End (q) on the "The program finished" bit.

Answer 3

Probably this could be of interest to you: Traceit