Is there an easy way to tell how much time is spent waiting for the Python GIL?

Question

I have a long-running Python service and I'd like to know how much cumulative wall clock time has been spent by any runnable threads (i.e., threads that weren't blocked for some other reason) waiting for the GIL. Is there an easy way to do this? E.g., perhaps I could periodically dump some counter to its log file.

My underlying motivation is to rule out the GIL as a source of mystery response latency from these long-running processes. There is no particular reason to suspect the GIL (other than that it would fit the symptoms), but other forms of logging haven't turned up anything yet, so, if it is easy, it would be nice to have this information.

Answer 1

I don't think there's an easy way. There's probably an awkward way, involving rebuilding Python to traverse the PyThreadState list and count the threads each time the lock is acquired, but I doubt it's worth the effort!

I know this is a speculative question but if you are even moderately concerned about there being delays caused by threading it may be prudent to move to a multiprocessing model instead of a multithreading model. Since processes are both safer and more scalable in Python they are almost always the best choice if practical.