C++/Java Performance for Neural Networks?

Question

I was discussing neural networks (NN) with a friend over lunch the other day and he claimed the the performance of a NN written in Java would be similar to one written in C++. I know that with 'just in time' compiler techniques Java can do very well, but somehow I just don't buy it. Does anyone have any experience that would shed light on this issue? This page is the extent of my reading on the subject.

Answer 1

The Hotspot JIT can now produce code faster than C++. The reason is run-time empirical optimization.

For example, it can see that a certain loop takes the "false" branch 99% of the time and reorder the machine code instructions accordingly.

There's lots of articles about this. If you want all the details, read Sun's excellent whitepaper. For more informal info, try this one.

Answer 2

I'd be interested in a comparison between Hotspot JIT and profile-guided optimization optimized C++.

The problem I see with the Hotspot JIT (and any runtime-profile-optimized JIT compiler) is that statistics must be kept and code modified. While there are isolated cases this will result in faster-running code, I doubt that profile-optimized JIT compilers will run faster than well optimized C or C++ code in most circumstances. (Of course I could be wrong.)

Anyway, usually you're going to be at the mercy of the larger project, using the same language it is written in. Or you'll be at the mercy of the knowledge base of your co-workers. Or you'll be at the mercy of the platform you are targetting (is a JVM available on the architecture you're targetting?). In the rare case you have complete freedom and you're familiar with both languages, do some comparisons with the tools you have at your disposal. That is really the only way to determine what's best.

Answer 3

This is not strictly about C++ vs Java performance but nonetheless interesting in that regard: A paper about the performance of programs running in a garbage collected environment.

Answer 4

The only possible answer is: make a prototype and measure for yourself. If my experience is of any interest, Java and C# were always much slower than C++ for the kind of work I was doing - I believe mostly because of the high memory consumption. Of course, you can come to a completely different conclusion.

Answer 5

If excessive garbage collection is a concern, you can always reuse unused high-churn objects.

Create a factory that keeps a queue of SoftReferences to recycled objects, using those before creating new objects. Then in code that uses these objects, explicitly return these objects to the factory for recycling.

Answer 6

In the grand scheme of things, you're debating maybe a 5% performance difference where you'd get several orders of magnitude increase by moving to CUDA or dedicated hardware.