Trouble with Emacs pdb and breakpoints in multi-threaded Python code

Question

I am running Emacs 23.2 with python.el and debugging some Python code with pdb.

My code spawns a sibling thread using the threading module and I set a breakpoint at the start of the run() method, but the break is never handled by pdb even though the code definitely runs and works for all intents and purposes.

I was under the impression that I could use pdb to establish breakpoints in any thread, even though full multi-threaded debugging is in fact not supported.

Am I wrong in assuming pdb within an M-x pdb invocation can break within any thread? If you don't believe me try this minimal example for yourself.

import threading

class ThreadTest(threading.Thread):
    def __init__(self,):
        threading.Thread.__init__(self)

    def run(self):
        print "Type M-x pdb, set a breakpoint here then type c <RET>..."
        print "As you can see it does not break!"

if __name__ == '__main__':
    tt = ThreadTest()

    tt.start()

Thanks to Pierre and the book text he refers to, I tried the option to include pdb.set_trace() as follows:

def run(self):
    import pdb; pdb.set_trace()
    print "Set a breakpoint here then M-x pdb and type c..."

But this only breaks and offers pdb controls for step, next, continue etcetera, if it is executed from a console and run directly within the Python interpreter, and crucially not via M-x pdb - at least with my Emacs and pdb configuration.

So my original question could do with being rephrased:

Is there a way to invoke a Python program from within Emacs, where that program uses inlined invocation of pdb (thereby supporting breaks in multi-threaded applications), and for there to be a pdb comint control buffer established auto-magically?

OR

If I run my Python application using M-x pdb and it contains an inline invocation of pdb, how best to handle the fact that this results in a pdb-session-within-a-pdb-session with the associated loss of control?

Thanks.

Answer 1

See http://heather.cs.ucdavis.edu/~matloff/158/PLN/ParProcBook.pdf, there's a section on multithreaded debugging.

3.6.1 Using PDB to Debug Threaded Programs
Using PDB is a bit more complex when threads are involved. One cannot, for instance, simply do something
like this:
pdb.py buggyprog.py
because the child threads will not inherit the PDB process from the main thread. You can still run PDB in
the latter, but will not be able to set breakpoints in threads.
What you can do, though, is invoke PDB from within the function which is run by the thread, by calling
pdb.set trace() at one or more points within the code:
import pdb
pdb.set_trace()
In essence, those become breakpoints.
For example, in our program srvr.py in Section 3.1.1, we could add a PDB call at the beginning of the loop
in serveclient():
while 1:
import pdb
pdb.set_trace()
# receive letter from client, if it is still connected
k = c.recv(1)
if k == ’’: break
You then run the program directly through the Python interpreter as usual, NOT through PDB, but then the
program suddenly moves into debugging mode on its own. At that point, one can then step through the code
using the n or s commands, query the values of variables, etc.
PDB’s c (“continue”) command still works. Can one still use the b command to set additional breakpoints?
Yes, but it might be only on a one-time basis, depending on the context. A breakpoint might work only once,
due to a scope problem. Leaving the scope where we invoked PDB causes removal of the trace object. Thus
I suggested setting up the trace inside the loop above.

Answer 2

Are you using the default python.el? I've given up on that and started using python-mode.el. Then type M-x shell, from the prompt type python myproblem.py (replace with your program name of course) and it will stop at the set_trace line. It works out of the box with pdb integration. (And it works on your program).