.NET runtime vs. Java Hotspot: Is .NET one generation behind?

Question

According to the information I could gather on .NET and Java execution environment, the current state of affairs is follows:

• Modern Java VM are capable of performing continuous recompilation, which combined with profiling can yield great performance improvements. Older JVMs employed JIT. More information in this article: http://www.ibm.com/developerworks/library/j-jtp12214/ and especially: Java theory and practice: Dynamic compilation and performance measurement

• .NET uses JIT or NGEN to generate native code, but once the native code is generated, no further (runtime) optimizations are performed.

Benchmarks aside and with no intention to escalate holy wars, does this mean that Java Hotspot VM is one generation ahead of .Net. Will these technologies employed at Java VM eventually find its way into .NET runtime?

Answer 1

Apparently someone was working on something similar for Rotor. I don't have access to IEEE so I can't read the abstract.

Dynamic recompilation and profile-guided optimisations for a .NET JIT compiler

Quote from Summary...

An evaluation of the framework using a set of test programs shows that performance can improve by a maximum of 42.3% and by 9% on average. Our results also show that the overheads of collecting accurate profile information through instrumentation to an extent outweigh the benefits of profile-guided optimisations in our implementation, suggesting the need for implementing techniques that can reduce such overheads.

Answer 2

I've never benchmarked the two to compare, and I'm more familiar with the Sun JVM, I can only speak in general terms about JITs.

There are always tradeoffs with optimizations, and not all optimizations work all the time. However, here are some modern JIT techniques. I think this can be the beginning of a good conversation if we stick to the technical stuff:

• escape analysis
• intrinsics
  • http://bugs.sun.com/view_bug.do?bug_id=6823354
  • http://weblog.ikvm.net/CommentView.aspx?guid=0404dd8a-88a8-4d62-9bcb-98324d57a2a9
• tail-call optimization
• on-stack replacement
• lock coarsening
• lock elision
• multi-threaded garbage collection
• low-pause garbage collection
• polymorphic method call removal
• fast heap allocation

There's also features that are helpful as far as good implementations of a VM go:

• being able to pick between GC
• implementations customization of each GC
• heap allocation parameters (such as growth)
• page locking

Based on these features and many more, we can compare VMs, and not just "Java" versus ".NET" but, say, Sun's JVM versus IBM's JVM versus .NET versus Mono.

For example, Sun's JVM doesn't do tail-call optimization, IIRC, but IBM's does.

Answer 3

They follow two different strategies. I do not think one is better than the other.

• .NET does not interpret bytecode, so it has to JIT everything as is gets executed and therefore cannot optimise heavily due to time constraints. If you need heavy optimizations in some part of the code, you can always NGEN it manually, or do a fast but unsafe implementation. Furthermore, calling native code is easy. The approach here seems to be getting the runtime good enough and manually optimise bottlenecks.

• Modern JVMs will usually interpret most of the code, and then do an optimized compilation of the bottlenecks. This usually gets better results than straight JIT'ing, but if you need more, you don't have unsafe in Java, and calling native code is not nice. So the approach here is to do as much automatic optimising as possible, because the other options are not that good.

In reality Java applications tend to perform slightly better in time and worse in space when compared to .NET.

Answer 4

You may be interested in SPUR which is a Tracing JIT compiler. The focus is on javascript but it operates on CIL not the language itself. It is a research project based on Bartok not the standard .NET VM. The paper has some performance benchmarks showing 'it consistently performs faster than SPUR-CLR' which is the standard 3.5 CLR. There haven't been any announcements about it's future relating to the current VM however. Traces can cross method boundaries which is not something HotSpot does AFAIK, JVM tracing JITs are mentioned here.

I'd be hesitant to say the .NET VM is a generation behind especially when considering all the sub-systems, in particular generics. How the GC and DLR vs invokedynamic compare I'm unsure but there are lots of details about them at places like channel9.