python tracing a segmentation fault

Question

Hi, I'm developing C extensions from python ad I obtain some segfaults (inevitable during the development...).

I'm searching a way to display at which line of code the segfault happens (an idea is like tracing every single line of code), how I can do that?

Answer 1

Here's a way to output the filename and line number of every line of Python your code runs:

import sys

def trace(frame, event, arg):
    print "%s, %s:%d" % (event, frame.f_code.co_filename, frame.f_lineno)
    return trace

def test():
    print "Line 8"
    print "Line 9"

sys.settrace(trace)
test()

Output:

call, test.py:7
line, test.py:8
Line 8
line, test.py:9
Line 9
return, test.py:9

(You'd probably want to write the trace output to a file, of course.)

Answer 2

If you are on linux, run python under gdb

gdb python
(gdb) run /path/to/script.py
## wait for segfault ##
(gdb) backtrace
## stack trace of the c code

Answer 3

Segfaults from C extensions are very frequently a result of not incrementing a reference count when you create a new reference to an object. That makes them very hard to track down as the segfault occurs only after the last reference is removed from the object, and even then often only when some other object is being allocated.

You don't say how much C extension code you have written so far, but if you're just starting out consider whether you can use either ctypes or Cython. Ctypes may not be flexible enough for your needs, but you should be able to link to just about any C library with Cython and have all the reference counts maintained for you automatically.

That isn't always sufficient: if your Python objects and any underlying C objects have different lifetimes you can still get problems, but it does simplify things considerably.