How to debug deadlock with python?

Question

I am developing a multi-threading application, which is deadlocking.

I am using Visual C++ Express 2008 to trace the program. Once the deadlock occurs, I just pause the program and trace. I found that when deadlock occurs, there will be two threads called python from my C++ extension.

All of them use Queue in python code, so I guess the deadlock might caused by Queue. But however, once the extension goes into python code, I can't see nothing but asm code and binary from the VC++ debugger.

I would like to know are there any way to dump the call stack of python code after I paused the program? And how can I know what lock are there in threads caused the deadlock?

Answer 1

If you can compile your extension module with gcc (for example, by using Cygwin), you could use gdb and the pystack gdb macro to get Python stacks in that situation. I don't know if it would be possible to do something equivalent to pystack in Visual C++ Express, but you might get some ideas from the pystack macro implementation anyway.

Since you mention you only see asm/binary in the VC++ debugger, you should make sure you compile Python with debug symbols. If VC++ is still showing asm, it might be that you need to tell VC++ where the source files are (sorry, haven't used VC++ in years so I can't tell what exactly you might need to do if this was the case).

You might also get some important information by adding lots of logging calls to your code, both Python side and your C++ extension.

In any case, I am almost certain the deadlocks are not due to Queue, but your own code.