Google is sponsoring an Open Source project to increase the speed of Python by 5x.
Unladen-Swallow seems to have a good project plan
Why is concurrency such a hard problem?
Is LLVM going to solve the concurrency problem?
Are there solutions other than Multi-core for Hardware advancement?
LLVM is several things together kinda of virtual machine/optimizing compiler, you have different frontends that take the input in a particular language an output the result in a intermediate language. This intermediate output can be run with the virtual machine, but also can be used to generate a standalone executable.
The problem with concurrency is that it just recently has become quite common. Although it was being used for a long time in scientific computing. So while is it know how to program a scientific calculation program to achieve great performance, it is another completely different thing to write a mail user agent/word processor that can be good at performance. Also most of the current OS were being designed with uniprocessor in mind, and while work with multicore processors they may not be fully prepared.
The benefit of LLVM with respect to concurrency is that you have an intermediate output, and if in the next years there are advances in concurrency, then with updating your interpreter you instantly gain those benefits, but this is not so easy if you have a standalone executable. So LLVM doesn't solve the concurrency problem per se but it leaves an open door for future enhancements.
Sure there're more possible advances for the hardware like quantum computers, genetics computers, etc. But we have to wait for them to become a reality.
The switch to LLVM itself isn't solving the concurrency problem. That's being solved separately, by getting rid of the Global Interpreter Lock.
I'm not sure how I feel about that; I use threads mainly to deal with blocking I/O, not to take advantage of multicore processors (for that, I would use the multiprocessing module to spawn separate processes).
So I kind of like the GIL; it makes my life a lot easier not having to think about tricky synchronization issues.
LLVM takes care of the nitty-gritty of code generation, so it lets them rewrite Psyco in a way that's more general, portable, maintainable. That in turn allows them to rewrite the CPython core, which lets them experiment with alternate GCs and other things needed to improve python's support for concurrency.
In other words, LLVM doesn't solve the concurrency problem, it just frees up your hands so YOU can solve it.