Python 3.2 ALPHA is out.
From the Change Log, it appears the GIL has been entirely rewritten.
A few questions:
UPDATE:
I'm fairly new to Python. So all of this is new to my but I do at least understand that the existence of a GIL with CPython is a huge deal.
Question though, why does CPython not just clone the interpreter like Perl does in an attempt to remove the need for the GIL?
Is having a GIL good or bad? (and why).
Neither -- or both. It's necessary for thread synchronization.
Is the new GIL better? If so, how?
Have you run any benchmarks? If not, then perhaps you should (1) run a benchmark, (2) post the benchmark in the question and (3) ask specific questions about the benchmark results.
Discussing the GIL in vague, handwaving ways is largely a waste of time.
Discussing the GIL in the specific context of your benchmark, however, can lead to a solution to your performance problem.
Question though, why does CPython not just clone the interpreter like Perl does in an attempt to remove the need for the GIL?
Read this: http://perldoc.perl.org/perlthrtut.html
First, PERL didn't support threads at all. Older PERL interpreters had a buggy module that didn't work correctly.
Second, the newer PERL interpreter has this feature.
The biggest difference between Perl ithreads and the old 5.005 style threading, or for that matter, to most other threading systems out there, is that by default, no data is shared. When a new Perl thread is created, all the data associated with the current thread is copied to the new thread, and is subsequently private to that new thread!
Since the PERL (only specific data is shared) model is different from Python's (all data is shared) model, copying the PERL interpreter would fundamentally break Python's threads. The PERL thread model is fundamentally different.
The best explanation I've seen as to why the GIL sucks is here:
http://www.dabeaz.com/python/GIL.pdf
And the same guy has a presentation on the new GIL here:
http://www.dabeaz.com/python/NewGIL.pdf
If that's all that's been done it still sucks - just not as bad. Multiple threads will behave better. Multi-core will still do nothing for you with a single python app.
Is the new GIL better? If so, how?
Well, it at least replaces op-count switching to proper time-count. This does not increase overall performance (and could even hurt it due to more often switching), but this makes threads more responsive and eliminates cases when ALL threads get locked if one of them uses computation-heavy single op-code (like call to external function which does not release GIL).
why does CPython not just clone the interpreter like Perl does in an attempt to remove the need for the GIL?
GIL is complex issue, it should not be viewed as Ultimate Evil. It brings us thread-safety.
As for perl, perl is a) dead, b) too old. Guys at Google are working on bringing LLVM goodies to CPython, which, among others, will improve GIL behavior (no complete GIL removal yet, tho): http://code.google.com/p/unladen-swallow/