Language performance

Question

Everyday discussions like "C# vs Java performance", "F# vs C# performance", "C vs C++", "native C++ vs C#" always are interesting. Just like idiom.com/~zilla/Computer/javaCbenchmark.html , or mb manageability.org/blog/archive/20030520%23p_the_problem_with_cameron

Generally, professional of language A proves that language A is faster than B.

In most of cases this test means that A's pro does know know how to implement benchmark problem fast in B, or B's utiliteis lybrarlies are not perfectly optimized.

What is language performance? The case is clear, when we are talking about different technologies. For example we can figure out when Nvidia CUDA is faster than CPU calculations and vice versa. Or analog computer is better than digital.

Everything is a bit harder, when we deal with languages with the same purpose. For example, I'm a C# programmer for several years, almost every day I was writing unsafe code and I've read Jeffrey Richter's book on low level memory management. I can implement some numerical algorithms in pinned memory scrupulously verifing every step of an algorithm. But I do not have in-depth knolege of Java. I'll implement test case superficially. And my benchmark will show that C# is faster many times. But it won't be true benchmark.

And finally, it's very hard to compare different languages with the same base. Like F# and C#. Just like this discussion

All test cases I've found just show that test's author do know language A and do not know language B.

So. What would be correct metric of what is faster?

PS. I really belive that C# is faster than F#, because I cannot imagine a task solvable in F#, which I cannot do in C# better (:

Answer 1

I think i would go something like:

ROC * ( HDT * HDD^2 * LOC^(1/2) )^(-1) * (ln(HFII / HFIB) + 1)
ROC - readability of code
HDT - hours of development time ( or hard is do develop n amount of code )
HDD - hours of debug time ( or how hard is to debug n amount of code )
LOC - lines of code
HFII - how fast it is
HFIB - how fast it needs to be

This is problem specific metric. So you would need to get average over several typical problems.

Any suggestions to make formula better are welcome.

Answer 2

Also entirely depends on your field and bottlenecks. I deal with web requests a lot in my programs which are always the slow point. I also don't care for fast code if it means I have to spend 3 days reading a single function when it needs maintenance.

Answer 3

The performance of language A vs the performance of language B is almost always a flawed comparison. What is actually being compared is the performance of an implementation of program P in an implementation of language A against the performance of an implementation of program P in an implementation of language B.

In other words, languages cannot be benchmarked for performance; programs can, and language implementations can.

These comparisons assume the program P is written equivalently in both languages, and that the implementations of both languages are equivalent.

The different implementations of a language may have better or worse performance relative to one another. For example, Google are working on optimizing Python to improve performance relative to the main stream interpreter, which has different performance characteristics to the Iron Python, Jython, Stackless, and PyPy implementations of python.

Ease of programming is such a horribly subjective thing to measure, as it is incredibly hard to find programmers equally familiar with language A and B, and who don't write the program better the second time based on the experience of writing the program the first time.

Various objective measures are possible, such as lines of code; but these are often flawed, leading to code golf.

Answer 4

What would be correct metric of what is faster?

Find a pro in language X and one pro in language Y. Devise some divine way of ensuring that they know just as much about optimization in their language as the other. Have them both solve the problem, then measure the difference between the two solutions.

IMNSHO, any blanket statement about "what is faster" are bound to be biased.

And there are so many other factors going into the "return on investment" calculation... availability of development tools for the target language on the target platform, availability of people qualified in the target language, productivity when coding in the language...

Answer 5

I think some of you forgot to mention one main thing, you cannot compare low and high level languages at all! ie. C# and C. In C you can write your own way which many times if you are good in what you doing going to be faster for your special needs that using already made in C# and fully rely on it. Again it doesn't mean you should avoid great functionality C# has which will make you programming much faster when later you can optimize specific things with C++ or C if you will feel that C# lacks in performance in certain area.

Now other debate is strict typing and dynamic one, obviously strict ie. Java or C# will require more line of code written in order to achieve same with let's say Python which is dynamic. I prefer strict as it eases on debugger and many pre building errors are detected.

For your question, on C you could probably write application that will be faster than same application written in C# ( depends if you are able to write better code than C# and .Net developers ) but it will require more time which another aspect of performance.