call multiple c++ functions in python using threads

Question

Suppose I have a C(++) function taking an integer, and it is bound to (C)python with python api, so I can call it from python:

import c_module
c_module.f(10)

now, I want to parallelize it. The problem is: how does the GIL work in this case? Suppose I have a queue of numbers to be processed, and some workers (threading.Thread) working in parallel, each of them calling c_module.f(number) where number is taken from a queue.

The difference with the usual case, when GIL lock the interpreter, is that now you don't need the interpreter to evaluate c_module.f because it is compiled. So the question is: in this case the processing is really parallel?

Answer 1

Threads currently executing the C extension code for which the GIL was explicitly released will run in parallel. See http://docs.python.org/c-api/init.html#thread-state-and-the-global-interpreter-lock for what you need to do in your extension.

Python threads are most useful for I/O bound execution or for GUI responsiveness. I wouldn't do python-heavy execution with threads. If you want guaranteed parallelism, check out the multiprocessing library.