What is the most effective way to present and communicate a performance improvement (e.g. percentages, raw data, graphics)?

Question

Is it better to describe improvements using percentages or just the differences in the numbers? For example if you improved the performance of a critical ETL SQL Query from 4000 msecs to 312 msecs how would you present it as an 'Accomplishment' on a performance review?

Answer 1

In currency. Money is the most effective medium for communicating value, which is what you're trying to use the performance review to demonstrate.

Person hours saved, (very roughly) estimated value of $NEW_THING_THE_COMPANY_CAN_DO_AS_RESULT, future hardware upgrades averted, etc.

You get the nice bonus that you show that you're sensitive to the company's financial position; a geek who can align himself with what the company is really about.

Answer 2

Use whichever appears most impressive given the change. According to one method of calculation, that change sped up the query by 1,300%, which looks more impressive than 13x improvement, or

• ============= <-- old query
• = <-- new query

Although the graph isn't a bad method.

If you can calculate the improvement in money, then go for that. One piece of software I wrote many years ago saved a few engineers a little bit of time each day. Figuring out the cost of salary, benefits, overhead and it turned into a savings of more than $12k per year for a small company.

Answer 3

Rule of the thumb: Whichever sounds more impressive.

If you went from 10 tasks done in a period to 12, you could say you improved the performance by 20% Saying you did two tasks more doesnt seem that impressive.

In your case, both numbers sound good, but try different representations and see what you get!

Sometimes graphics help a lot of the improvement is there on a number of factors, but the combined somehow does not look that cool

Example: You have 5 params A, B, C, D, E. You could make a bar chart with those 5 params and "before and after" values side by side for each param. That sure will look impressive.

God im starting to sound like my friend from marketing!

runs away screaming

Answer 4

1. Take potato
2. Drench Potato in Lighter Fluid
3. Light potato on fire
4. Hand potato to boss
5. Make boss hold it for 4 seconds.
6. Ask boss how long those 4 seconds felt
7. Ask boss how much better half a second would have been
8. Bask in glory

Answer 5

It is always better to measure relative improvement.

So, if you brought it down to 312ms from 4000ms then it is an improvement of 3688ms, which is 92.2% of the original speed. So, you reduced the runtime by 92.2%. In other words, you brought the runtime down to only 7.8% of what it was originally.

Absolute numbers, on the other hand, usually are not that good since they are not comparable. (If your original runtime was 4,000,000ms then an improvement of 3688ms isn't that great.)

Answer 6

See this link for some nice chart suggestions.

Comparison to Requirements

If I have requirements (response time, throughput), I like to color code the absolute numbers like so:

Green: <= 80% of the requirement (response time); >= 120% of > the requirement (throughput)
No formatting: Meets the requirement.
Red: Does not meet the requirement.

Comparisons are interesting, but only if we have enough to see trends over time; Is our performance steadily improving or degrading? Ultimately, the business only cares if we're meeting the requirement. It's only when we don't that they ask for comparisons to previous releases.

Comparison of Benchmarks

If I'm comparing benchmarks to some baseline, then I like to use percentages, but only if the benchmark is a statistically significant change from the baseline.

Hardware Sizing

If I'm doing hardware sizing or capacity planning, then I like to express the performance as the absolute number plus the cost per transaction. For example:

• System A: 1,000 transactions/second, $0.02/transaction
• System B: 1,500 transactions/second, $0.04/transaction

Answer 7

you can make numbers and graphs say anything you want - the important thing is to make them say something meaningful and relevant to the audience you're presenting them to. if it's end users you can show them differences in the screen refreshes (something they understand), to managers perhaps the reduced number of servers they'll need in order to support the application ($ savings), financial...it's all about the $ how much did it save them. a general rule is the less technical the group the more graphical and dramatic you need to be.