tags:

views:

472

answers:

6

What is a global interpreter lock and why is that an issue?

A lot noise has been made around removing the GIL from Python, and I'd like to understand why that is so important. I never wrote a compiler nor an interpreter myself, so don't be frugal with details, I'll probably need them to understand.

+5  A: 

Suppose you have multiple threads which don't really touch each other's data. Those should execute as independently as possible. If you have a "global lock" which you need to acquire in order to (say) call a function, that can end up as a bottleneck. You can wind up not getting much benefit from having multiple threads in the first place.

To put it into a real world analogy: imagine 100 developers working at a company with only a single coffee mug. Most of the developers would spend their time waiting for coffee instead of coding.

None of this is Python-specific - I don't know the details of what Python needed a GIL for in the first place. However, hopefully it's given you a better idea of the general concept.

Jon Skeet
Like the coffe mug analogy.
e-satis
+16  A: 

Python's GIL is intended to serialize access to interpreter internals from different threads. On multi-core systems, it means that multiple threads can't effectively make use of multiple cores. (If the GIL didn't lead to this problem, most people wouldn't care about the GIL - it's only being raised as an issue because of the increasing prevalence of multi-core systems.) If you want to understand it in detail, you can view this video or look at this set of slides. It might be too much information, but then you did ask for details :-)

Note that Python's GIL is only really an issue for CPython, the reference implementation. Jython and IronPython don't have a GIL. As a Python developer, you don't generally come across the GIL unless you're writing a C extension. C extension writers need to release the GIL when their extensions do blocking I/O, so that other threads in the Python process get a chance to run.

Vinay Sajip
Whatching the video. Great stuff.
e-satis
Good answer - basically it means that threads in Python are only good for blocking I/O; your app will never go above 1 CPU core of processor usage
Paul Betts
+1  A: 

Wikipedia has a nice description of a global interpreter lock

http://en.wikipedia.org/wiki/Global%5FInterpreter%5FLock

That article links this nice article that discusses the GIL in Python.

http://www.ddj.com/linux-open-source/206103078?pgno=2

Greg
Possibly it's better to link to the start of that DDJ article - http://www.ddj.com/linux-open-source/206103078
Vinay Sajip
Thanks, but I did my homework before asking the question here, and of course went to google and wikipedia first.
e-satis
+1  A: 

Whenever two threads have access to the same variable you have a problem. In C++ for instance, the way to avoid the problem is to define some mutex lock to prevent two thread to, let's say, enter the setter of an object at the same time.

Multithreading is possible in python, but two threads cannot be executed at the same time at a granularity finer than one python instruction. The running thread is getting a global lock called GIL.

This means if you begin write some multithreaded code in order to take advantage of your multicore processor, your performance won't improve. The usual workaround consists of going multiprocess.

Note that it is possible to release the GIL if you're inside a method you wrote in C for instance.

The use of a GIL is not inherent to Python but to some of its interpreter, including the most common CPython. (#edited, see comment)

The GIL issue is still valid in Python 3000.

poulejapon
Stackless still has a GIL. Stackless does not improve threading (as in, the module) - it offers a different method of programming (coroutines) which attempt to side-step the issue, but require non-blocking functions.
jnoller
Good point. Thank you for the comment
poulejapon
+2  A: 

Watch David Beazley tell you everything you ever wanted to know about the GIL.

hughdbrown
+1  A: 

Here's a longish article talking about the GIL and threading in Python I wrote awhile back. It goes into a fair amount of detail on it:

http://jessenoller.com/2009/02/01/python-threads-and-the-global-interpreter-lock/

jnoller