Are there scenarios where JIT compiler is faster than other compilers like C++?
Do you think in the future JIT compiler will just see minor optimizations, features but follow a similar performance, or will there be breakthroughs that will make it infinitely superior to other compilers?
It looks like the multi core paradigm has some promise but it's not universal magic.
Any insights?
Yes, there certainly are such scenarios.
I think there will be breakthroughs in the future. In particular, I think that the combination of JIT compilation and dynamic typing will be significantly improved. We are already seeing this in the JavaScript space with Chrome's V8 and TraceMonkey. I expect to see other improvements of similar magnitude in the not-too-distant future. This is important because even so-called "statically typed" languages tend to have a number of dynamic features.
JIT compilers have more data they can use to influence optimizations. Of course, someone actually has to write code to use that data, so it's not a simple as that.
Basically, JIT compilers has a chance to actually profile the application being run, and do some hinting based on that information. "offline" compilers will not be able to determine how often a branch jumps and how often it falls through, without inserting special code, ask the dev to run the program, put it through its paces, and recompile.
Why does this matter?
//code before
if(errorCondition)
{
//error handling
}
//code after
Gets converted into something like:
//code before
Branch if not error to Code After
//error handling
Code After:
//Code After
And x86 processors would not predict a conditional jump ahead in the absence of information from the branch prediction unit. That means that it predicts the error handling code to run, and the processor's going to have to flush the pipeline when it figures out that the error condition didn't occur.
A JIT compiler could see that, and insert a hint for the branch, so that the CPU would predict in the correct direction. Granted, offline compilers can structure the code in a way that would avoid the mispredict, but if you ever need to look at the assembly, you might not like it jumping around everywhere....
Yes, JIT compilers can produce faster Machine Code optimized for the current environment. But practically VM programs are slower than Native programs because JITing itself consumes time (more Optimization == more time), and for many methods JITing them may consume more time than executing them. And that's why GAC is introduced in .NET
A side effect for JITing is large memory consumption. However that's not related to computation speed, it may slow down the whole program execution, because large memory consumption increases the probability that your code will be paged out to the secondary storage.
Excuse me for my bad English.
JIT has advantages, but I don't see it taking over completely. Conventional compilers can spend more time optimizing, while a JIT needs to strike a balance between too much optimization (taking more time than is saved by optimization) and too little (taking too much time in straight execution).
The obvious answer is to use each where it's superior. JITs can take advantage of run-time profiling more easily than conventional optimizers (although there are compilers that can take run-time profiles as input to guide optimization), and can generally afford to do more CPU-specific optimizations (again, lots of conventional compilers do this, but if you expect to run the executable on different systems they can't take full advantage of it). Conventional compilers can spend more time, and do it in different ways.
Therefore, the language system of the future will have good optimizing compilers that will emit executable code designed for use by good optimizing JIT compilers. (This is also, for many people, the language system of the present.) (The language system of the future will also support everything from modern Python/VB scripting to the ugliest high-speed number crunching.)
As with many things, this was foreshadowed by Lisp. Quite some time ago, some Lisp systems (can't really say many, there haven't been all that many Common Lisp implementations) interpreted Lisp functions by compiling them on the fly. Lisp S-expressions (what code is written in) are fairly straightforward descriptions of parse trees, so compilation could go pretty fast. In the meantime, an optimizing Lisp compiler could crunch the code where performance was really important ahead of time.
Another thing that was skipped in this conversation is that when you JIT a piece of code it can be compiled to a free spot in memory. In a language like C++ if the DLL is based such that that piece of memory is not available it will have to go through the expensive process of rebasing. It is faster to JIT code into an unused address then rebase a compiled DLL into a free memory space. To make things worse, a rebased DLL can no longer be shared. (see http://msdn.microsoft.com/en-us/magazine/cc163610.aspx)
I haven't been very impressed with some of the optimizations in C# 3.5 JIT code. Simple things like bit twiddling that is necessary for compression are horribly inefficient (it refused to cache values in a CPU register and instead went to memory for every operation). I don't know why it would do this but it makes a huge difference and there is nothing I can do about it.
Personally I think a good solution would be an Optimization Level (1-100) that you could set to tell JIT compiler how much time you think it ought to spend optimizing your code. The only other solution would be an AOT (Ahead of Time) compiler and then you lose many of the advantages of JIT code.
JIT compilers know more of the systems then static compilers. Adding multitheading on the fly specific to the machine can be a gigantic speed imporvement once they get it working.
JIT compilers in general have a bit of startup latency where the first run of the program/code can be 'MUCH' slower then precompiled code. The cold start disadvantage.
Another big advantage of JIT compilation is that the compiler can be updated after your program has been build and gain new compiler tricks without needing a complete new program deployment.