Allowing threads from python after calling a blocking i/o code in a python extension generated using SWIG

Question

I have written a python extension wrapping an existing C++ library live555 (wrapping RTSP client interface to be specific) in SWIG. The extension works when it is operated in a single thread, but as soon as I call the event loop function of the library, python interpreter never gets the control back. So if I create a scheduled task using threading.Timer right before calling the event loop, that task never gets executed once event loop starts. To fix this issue, I added Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS macros manually in the SWIG auto generated wrapper cxx file around every doEventLoop() function call. But now, I want to do the same (i.e. allow threads) when SWIG generates the code itself and not to change any code manually. Has anyone done something similar in SWIG?

P.S. - I would also consider switching to any other framework (like SIP) to get this working. I selected SWIG over any other technology is because writing SWIG interface was really very easy and I just had to include the existing header files.

Answer 1

SWIG gives you plenty of hooks to help make this happen. If a coarse solution is sufficient for your needs, one thing I've done in the past is put something like this in my .swig file:

%exception {
    Py_BEGIN_ALLOW_THREADS
    $action
    Py_END_ALLOW_THREADS
}

This (ab)uses the SWIG facility for decorating C function calls with some kind of error-handling logic in order to decorate those calls with a GIL unlock/lock. See Exception handling with %exception in the SWIG docs for details on what's going on here.