Are there benchmarks on how bad the performance regression in Perl 5.10.0 was?

Question

I am working in a high availability environment at the moment, so performance is a bit of an issue for this company. I found out today they are running Perl 5.10.0, which according to perl5101delta has a performance regression in list assignments. Now since we're on Debian, updating isn't exactly easy, so I am looking for statistics to make a point for how much of an improvement updating would be for us to update.

Answer 1

Other people's code benchmarks would only show what the difference is for these other people's code.

I assume that you already have a number of benchmarks which you use to quantify the performance of your "high availability environment", which may show the current number of requests per second that can be done to a specific page (for a website) or the number of subroutine calls for a specific high-visibility subroutine in an important module.

If you don't already have something like that, I would suggest creating some code which tracks the current performance of the system as it is, especially for the bits that you deem are the most high profile or for which a degraded performance would have a negative impact for the company.

Bear also in mind that a lot of the code may be IO-bound, and the regression you're referring to may be negligible with regards to it: it regards list assignments so it impacts most subroutine calls and parameters juggling. If your most precious code spends a long time waiting for the database to report back data, the above regression should be negligible for you.

A good way to find out which is which, for your own code, is to benchmark it and measure its current performance.

Once you have that, and completely unrelated from the specific versions you'd like to migrate either from or to, you'll be able to install any version you'd like to migrate to using App::perlbrew, install the modules you need to get the benchmarks going with App::cpanm, and then "simply" run your benchmark/test suite.

As an added bonus, you'll also be able to track the development versions of Perl and provide valuable feedback to perl5-porters for regressions that at that point may be able to be fixed before the next major release.

Although the perl5-porters do go to great length to ensure existing code doesn't break, that deprecations are sanely done every couple major releases, etc. things do break and the only way to know whether they do or do not break for your own code is to test it.

Added: To address the specific question, regarding Perl's "measurable performance drop in list assignment" regression, you could install various Perls via perlbrew and benchmark them with:

use Benchmark qw/:all/;
sub test_this {
  my ($a,$b,@c) = @_;
  1;
}
timethis(10_000_000, "test_this(1..10);");

Do that for Perl 5.10 and Perl 5.10.1 or other perls and see.

For my machine, 5.10.1 yields 540k/sec, whereas 5.10.0 yields 498k/sec.