Benchmarks of scripting languages doing the same task?

Question

This is not a X vs Y question - do not reply as same.

Does anyone know where I could find reviews or reports on tasks that people did in two or more high-level scripting languages to see which one was more suited for a certain job? (note that this is different from looking for a language suited for all jobs!) I really want to know which types of operations are better suited to which languages so that I can make the most of them.

For example: sockets, the file system, logic evaluation, regex, or drawing.

Given my options I'm mostly interested in Python, PHP, Perl, and Ruby.

Answer 1

The best thing to do would be to create your own benchmarks for the specific tasks you are interested in. Pick languages that you like working in and then write benchmarks for the system you will be using and the task you will be performing. If you are very concerned about speed I would also recommend looking at the individual operations and how to optimize them in each of the languages (examples: argument order, memory usage...)

Answer 2

There is always the good old Alioth Shootout site.

But since you have chosen languages which are in the "dead-slow" category unless you hit their strengths.

There is absolutely nothing wrong with being in the "dead-slow" category is the language lets you express your intentions in a concise and clear way, and if the processing is not your bottleneck (it usually is'nt and if it is it is again usually easily fixed by recoding the offending piece, provided you kept things loosely coupled.

For this make a performance budget, subtract all things external to your code, see what remains. Now code a couple of things which you worry about and see if they fit in the budget. You'll be surprised what you can do in 30ms if you do not need to wait on database or remote connections.

Answer 3

There's the programming language shootout: http://shootout.alioth.debian.org/ Although it may not measure enough of the things you're looking for.

However, benchmarks almost certainly won't tell you anything useful about high level ideas of the sort you listed. For those things, the performance (as in speed of execution) has almost nothing to do with what language the code is written in (at least for the set of languages you listed, all of which have quite similar execution models), and almost everything to do with how the code is structured.

If you want to be able to choose one of the languages you listed (for a particular task) based on something relevant, the questions you should be asking are:

• What restrictions are there given the platform you want to run on? All of the languages you listed have large runtime systems -- interpreters -- which means they may have heavy deployment costs if you want to use them on systems on which they are not already installed. Having said that, Python at least has 'py2exe' and 'py2app' which can be used to make deployable executables of your application for Windows or Mac -- there may well be something similar for the other languages you listed.
• What relevant functionality is provided in the standard library for the language? (in this case, all the listed languages have very large, useful standard libraries).
• What relevant functionality is available in external libraries that you could use?
• How comfortable are you using the language?

Finally, if execution speed really is very important in your application, and if you find that you can't achieve the speed you need even with an appropriate choice of architecture in your chosen language, then you can identify the bottleneck and re-implement that in a compiled language, providing an interface back to the scripting language so it can be integrated into the application. All of the languages you listed support such native-code extensions.

Answer 4

I would look at the programming tasks on Rosetta code.

You should be able to find idiomatic code for the languages you're interested in, and then benchmark under whatever usage conditions you expect. This frees you from having to write the tests yourself, and you often see interesting approaches for the different languages.