How is the current performance of the Mono virtual machine?

Question

The web is full of different kinds of performance test of different languages, compilers, and virtual machines. Yet hardly any of these test the performance using some real-world scenario. In addition, searching Google for these kind of benchmarks usually yields mainly only several years-old benchmarks as those have had the most time to gather links to them.

Do any of you have some real insight into the current performance of different virtual machines? In addition I'd especially like to know how Mono's performance compares to those of Microsoft's .Net and Sun's Java and how the performance of different virtual machines has evolved lately.

Answer 1

For a comparison of Java and Mono you can take a look at The Computer Language Benchmarks Game.

Answer 2

While I haven't used Mono, I guess it depends on what you do with it. I can't give you exact numbers on things, but here's an interesting tidbit about Mono's floating point performance:

http://forums.xna.com/forums/p/24249/24249.aspx

As Mono allows the use of your CPU's SIMD instructions (SSE2 and SSE4 at the moment, I believe) to dramatically speed up floating point calculations, it can blow away .NET at this kind of thing (up to 10x faster), as the chart shows (and hopefully Microsoft will implement something similar soon, .NET 4.5, please?). However, the chart also shows that .NET is still significantly faster than Mono when not using Mono.Simd. And you could take a huge leap of faith and extrapolate that 20% difference in floating point performance to other areas, like string performance.

However, that is Mono 2.2 and things may have changed drastically as Mono is moving quite fast these days, or so I hear.

Answer 3

Measuring performance is a complicated matter. In the past, when languages were tested against the same operating system, running the same hardware and a very limited set of libraries it was possible to create benchmarks that could give a linear metric that would measure a system. It would let folks evaluate things from zero to ten, assimilate the result and move on quickly to the next subject.

Things have become more complicated with modern systems as there are multiple variables to take into account.

At least in Mono's case there are plenty of variables that come into play:

• Code:

  • The quality of the native code generated.
  • The speed at which the native code is generated.
  • The memory required to generate the code and to optimize the code
  • Is the code generator multi-threaded
  • Is the generated code thread-safe
  • Does it take advantage of CPU specific features, at compile time or JIT time.
  • Can it use SIMD instructions if available.
  • Does the language map itself neatly to multi-core platforms
  • Does the language provide enough parameters for an optimizier to tune your code automatically (Like Fortran does).

• Memory management:

  • The garbage collection algorithm used
  • Does the GC scale with multiple CPUs?
  • Is the GC incremental, or real time?
  • Does it support thread-local storage for improved performance?
  • Is it precise, compacting, generational, conservative and what mixes of each.

• API design:

  • Are the APIs designed for latency or bandwidth
  • Do APIs support automatically scaling to multiple CPUs.
  • Can you offload heavy duty work to a GPU?
  • Do your APIs support streaming interfaces

All of these things complicate matters very much and make a simple 0 to 10 answer very hard to give.

If you were to partition languages in classes, and you assume a competent and performance aware programmer, I would divide the world in these classes:

• Tier 1: Hand tuned assembly language by a professional
• Tier 2: Statically compiled, strongly typed languages: C/C++/Fortran/
• Tier 3: managed/JIT languages: Java/C#/.NET/Mono/Boo/F#
• Tier 4: dynamically typed/JITed languages: Google V8, IronPython, IronRuby
• Tier 5: pure interpreted languages: Python, Perl
• Tier 6: pure interpreted languages, with too many features for their own good.

But the languages do not paint an entire picture, the APIs that you will consume, the hosting operating system and other facilities will have a big impact on your results.

For example, recently in Mono we added support for replacing Mono's code gen engine with a more advanced, highly optimizing engine (the LLVM engine). It turns out that it was incredibly hard to find a test where the overhead of using LLVM was worth the extra memory use: desktop and web applications did not exhibit much of a difference. And this is probably due to the fact that these are mostly I/O bound applications.

Using LLVM was useful for scientific and computationally intensive applications, but in real life it did not make much of a difference from Mono's default optimization settings.

As for the specifics of Mono: although Mono does use Boehm's GC, what most folks do not realize is that Boehm can be configured in various ways. The default layman configuration is indeed not very powerful, but it works for everyone that wants a quick GC. Mono does not use Boehm in this mode, Mono configures Boehm extensively to work in precise-mode as well taking advantage of thread local storage, multi-core GC and release-memory-to-the-OS modes.

Answer 4

I know this is old, but I just found it and none of the current answers (even miguel's) address a fundamental flaw in your question: a virtual machine.

You seem to be mis-informed on this point. .Net does not use a VM, and neither does mono. It is true that .Net uses a runtime library, and the code does compile to IL for deployment similar to Java's bytecode. However, a runtime is not a virtual machine. The difference is that after deployment the IL is itself fully compiled to native machine code before execution. No virtual machine need apply.

Answer 5

I benchmarked Mono 2.0 and 2.2 earlier this year using SciMark2 and found that Mono's performance had increased slightly but it is still far slower than most other VMs.