Refactoring (and testing) for performance

Question

I have a library that provides a reflection API on top of describeType() (a method that returns an XML object with all the specs of a class or instance). Since this library is used in several other libraries and frameworks, I really want it to be as fast as possible.

The problem I'm facing is that I'm not sure about the best approach to refactor this libraries' code and make it possible to test and compare the results. I could easily add timers to the code, log the ouput and compare the results, but it seems like a lot of work to do this manually.

So this brings me to the following questions:

• Has anyone done something similar before?
• How did you test and compare the results of the changes?
• Is there perhaps any testing framework (also non-actionscript) that helps in performance testing in such a scenario?
• Do you have any other general tips?

Answer 1

You want a profiler. Your compiler should come with one, for example gprof comes with GCC - there's a tutorial here.

Answer 2

Measuring and finding are different problems.

You can make code fast by finding and fixing what takes the most time, and that is not measuring, it is sampling - the more time something takes, the fewer samples it takes to find it.

What measuring is good for is seeing how much time was actually saved.

It is important in sampling not to throw away information. The program counter alone (as in gprof) is not enough. You need at least the whole call stack to be sampled. It is important to look at line-level information, because summarizing at functions discards information. It is important to sample at random wall-clock time, not random CPU time (as in gprof and other profilers). Samples at random CPU time are blind to time spent in needless I/O or system functions.

A good profiler is RotateRight/Zoom. I use this technique.

Answer 3

You can use the profiler from FlashBuilder (not free) to see where things can be improved.

You can optimise at the bytecode level (without touching your source) using TDSI for example (still in development and more optimisation will come)

If you want to see what 's going under the hood and understand more the compiled code, look at the tamarin source code (Adobe VM use in the flash player) and learn the abc bytecode

Or write multiple function and measure their performance , do small refactor step by step to see what are the gain, you can use gskinner library for the measurement (Don`t use the debug player since some function are slower int it).

Read some paper on as3 optimisation. There are good people like Joa Ebert, Grant Skinner.

But if the gain after optimisation is small prefer to keep readability of your code.