tags:

views:

137

answers:

2

This is a follow up to http://stackoverflow.com/questions/1417473/call-python-from-c

At the startup of the programm I call the following function to initialize the interpreter:

void initPython(){
    PyEval_InitThreads();
    Py_Initialize();
    PyEval_ReleaseLock();
}

Every thread creates it's own data structure and acquires the lock with:

PyGILState_STATE gstate;
gstate = PyGILState_Ensure();
//call python API, process results
PyGILState_Release(gstate);

Rather straight forward once you understood the GIL, but the problem is that I get a segfault when calling Py_Finalize().

void exitPython(){
    PyEval_AcquireLock();
    Py_Finalize();
}

The reference is rather dubious about Py_Finalize() (or maybe I'm just reading it the wrong way) and I'm not sure if PyEval_AcquireLock() can acquire the lock if there are some active threads and what happens if there are active threads when Py_Finalize() is called.

Anyways, I get a segfault even if I'm sure that all threads have finished their work, but only if at least one was created. E.g. calling initPython() followed from exitPython() creates no error.

I could just ignore the problem and hope the OS knows what it does, but I'd prefere if I could figure out what is going on..

+1  A: 

Have you tried commenting out all the 'work' done in your threads? Replace it with a busy loop or a sleep or something. That will help pinpoint whether it is your initialisation/shutdown code, or something you're actually doing to Python in between. Perhaps you're not setting up the threads properly - there are a lot of thread-specific functions in the C API and I'm not sure which ones you need to ensure proper operation.

Kylotan
I've commented everything out but PyGILState_Ensure() and Release() and the error still occurs. If I comment them out as well there's no problem..
Voo
In that case, I'm guessing there's something about the thread management that is not being done properly. Unfortunately the relevant C API page with all the thread functions on it is far from obvious as to which of those calls you need.
Kylotan
A: 

Yeah the whole section is rather dubious but I think I've got my mistake.

I've got to save the PyThreadState when initializing the interpreter and swap this state back in when I finish it (no idea why I need a specific ThreadState to call Finalize - shouldn't every State work as well?)

Anyways the example if other people got the same problem:

PyThreadState *mainstate;

void initPython(){
    PyEval_InitThreads();
    Py_Initialize();
    mainstate = PyThreadState_Swap(NULL);
    PyEval_ReleaseLock();
}

void exitPython(){
    PyEval_AcquireLock();
    PyThreadState_Swap(mainstate);
    Py_Finalize();
}

The only problem with this is, that I can acquire the lock like every other thread, even if there are still threads working. The API doesn't mention what happens when Finalize() is called while other threads are still working. Sounds like the perfect example of a race condition..

Voo