I'm reading a presentation in which different programming languages are being compared. And one of characteristics which is measured is 'Language Performance'. What does it mean?
+3
A:
The real answer: it means nothing, or whatever the author of that presentation was thinking about at the time.
Maybe it means "how much time it takes to perform actions with the same semantics relative to other languages"? But who can say?
Jonathan Feinberg
2010-02-16 18:07:12
++. You might make a case for Performance of an Implementation of a Language. But languages don't really have performance characteristics of their own.
Dustman
2010-02-16 18:09:00
Actually I guess he meant either 'how much time you need to implement the same things in different languages' or 'how much time programs written in different languages need to calculate the same things using the same algorithms'.
Roman
2010-02-16 18:10:03
@Dustman: Actually, they have performance limitations sometimes. The classic example is numerics in Fortran 77 and C90; the fact that C90 had to worry about aliasing effects made things like matrix manipulation slower than comparable F77 compilers, which basically ignored such effects.
David Thornley
2010-02-16 18:18:39
@DavidT: Which is partly the point, I guess. How gets reduced to optimized processor instructions is where performance of a language on a given set of hardware is determined. If I were an optimistic person, I might assume we were talking about a canonical or typical implementation, but since the vast majority of writings (even in school!) are of the "Java is slow, C is fast" variety, I'm just not able to summon up the necessary good feelings to make that assumption. Technically you're right, of course, that a language spec might overly constrain an implementation.
Dustman
2010-02-16 19:44:27
A:
Some languages require large runtime environments in order to run properly, this mainly applies to interpreted languages like Python or Ruby. Many compiled languages have smaller runtime overhead, like Objective-C. These factors contribute to performance.
pokstad
2010-02-16 18:11:31
+1
A:
This is a loaded question and is definitely up for interpretation. If this is a high quality paper, it should should specify exactly how they measured it so the reader can make their own judgment on the validity of the numbers. How I would measure it though is:
I would create/find a series of programs that do exactly the same thing and can be written in very close to the same way in every language. It's possible there already exists a suite of software (a la SPEC cpu2006) that does exactly that. After finding these programs, I'd run them and measure the performance. Here, it's still hard to draw conclusions, as different languages (and/or runtimes) do different things better and worse, and it's very difficult to find a set of programs that "correctly" exercise all parts of the studied languages.
J Teller
2010-02-16 18:13:51
I'd also like to add that even on the same language different compilers and runtimes can run the same program with significantly different performance characteristics. All of that needs to be taken into account so the analysis is "fair."
J Teller
2010-02-16 18:37:17
Yeah. In C, for example, the slowest and fastest implementations differ by a factor of 100000 or more.
Jörg W Mittag
2010-02-16 20:44:06
+1
A:
Usually it means that the author prefers one of the languages being discussed, and is attempting to convince you that his preference is valid and based on solid facts rather than personal taste (or, arguably, lack thereof).
Jerry Coffin
2010-02-16 18:32:03
+2
A:
I would define language performance as meaning the performance of the best existing implementations of a language when provided with typical, idiomatic implementations of algorithms.
Some languages are much easier to write efficient implementations of than others. C, for example, has basically always been a "fast" language because it's very close to the metal and easy to write an efficient compiler for. As performance is technically a property of the implementation, a slow language can become a fast language as implementations improve. Java, for example, was interpreted in its early versions and was considered (rightly at the time) to be a "slow" language. Since then, the JIT compilers and garbage collectors have gotten so good that Java now rightfully deserves a place among "fast" languages. This illustrates why such comparisons of language performance need to be taken with a grain of salt.
dsimcha
2010-02-16 18:34:47
However, when talking about "best existing implementation", you have to be careful how you define *that*. For example, there are papers that show that garbage collection is as efficient as manual memory management, but there are other papers that show that virtual memory makes garbage collection 40% slower. Now, virtual memory is *only* needed for C, it doesn't even make sense for, say, Java or PHP or Ruby, so to be fair, the "best existing implementation" for one of those, must necessarily be one without virtual memory. Similar for CPU architectures: modern CPUs are *heavily* optimized for C
Jörg W Mittag
2010-02-16 20:40:21
... and are very bad at executing, say, Haskell or Erlang. There are *other* CPU architectures, which are very good at running Haskell or Erlang, on which C doesn't even *run*.
Jörg W Mittag
2010-02-16 20:41:33
Virtual memory was invented because C can access memory directly. If one C program assumes that it can write data to address `X` and another program assumes it can write *its* data to address `X`, those programs will crash. Therefore, virtual memory *pretends* that either of those programs is the only program in the entire universe and translates program A's address `X` and program B's address `X` into two different addresses. Java, PHP, Ruby, Python, ECMAScript, Perl etc. (IOW: all languages that are actually used in the real world) do not allow direct memory access, making VM unnecessary.
Jörg W Mittag
2010-02-17 16:19:55
The thing is: virtual memory is *really* expensive. It is so dog-slow that it even needs special hardware support to even be viable at all. But those MMUs are really expensive and complex, too, and they take up quite a bit of die real estate on the CPU. And they are still pretty slow, and in particular they induce huge latencies on context switches. The Microsoft Singularity team measured a 10% performance hit just from *turning on* the MMU, without even using it. (Actually using it increased the slowdown to over 30%.) Given that the vast majority of code is not written in C, it doesn't ...
Jörg W Mittag
2010-02-17 16:25:00
... make sense to have VM anymore, which in turn means in doesn't make sense to have MMUs anymore, which in turn means that our CPUs will instantly become faster, simpler, smaller, cheaper and our systems more responsive and faster, and garbage collection (which again, is what the vast majority of all programming languages created since 1959 uses) becomes as cheap as manual memory management. It's really a no-brainer, except that unfortunately the powers-that-be at Intel, AMD, Microsoft, Apple and Co. don't seem to have a brain.
Jörg W Mittag
2010-02-17 16:29:07
+1
A:
It's not, as far as I know, a term with a universally acknowledged definition. In fact. "Language Performance" probably means "some metric for which my favorite language has a good score". More seriously, the presentation should define the terms. If this term isn't defined in a comprehensible fashion, then the presentation lacks substance.
Liz Albin
2010-02-16 18:37:51