Proving the ROI of a technology?

Question

How does one prove the ROI of a technology to their manager?

The closest thing I have found to a document on how to do this is:

http://www.agilejournal.com/pdf/Finding-ROI-in-Build-Automation.pdf

There are formulas in this document, but I can't really tell if they are just alot of marketing or if they are accurate formulas on how to calculate ROI.

I'm not really trying to calculate the ROI of the build tool in the above paper, I was just trying to calculate the ROI of a simple build tool like ANT.

Answer 1

I can't imagine that there would be any sensible way to accurately measure ROI on developer tools and practices. The only thing I can think of where that would might be possible would be in factory environments where you you might be able to measure productivity and average quality.

I'd suggest doing what everyone else does, which is pick some formulas that will support what you want and then tweak them until the ROI is high enough to justify the investment.

Answer 2

They don't cut to the meat of the question: the intangible benefits - though they at least try to walk through an example. The formulas are just to get ROI as a nice percentage - if "using build tools" was a stock, how much return would I get on my investment?

Which already shows that the question itself is flawed: An automated build is mainly an instrument to improve quality; improving productivity is usually a secondary concern.

However, this doesn't help when talking to the guys sitting on the money.
Metrics I woud use to analyse effect of a build tool:

• Turnaround time from checkin to final media
• Number of builds (for testing, for release, ..)
• Number of build requested (with faster builds, you can expect an increase in demand)
• Number of errors introduced during manual build (assuming you track those)
• Number of developers able to publish a release
• Estimated resources (time, licences, build server, ..) for implementation and maintenance
• Analysis of low-probability, high risk scenarios

Often, an automated build tool pays for itself just by removing a bottleneck: every developer can publish the software, not just John the Builder.

The last point is important (but hardest to give numbers for), as the total cost of bugs doesn't have a normal distribution, but is highly "pareto": a single bug can give you some nasty press, or make key customers switch to competition.

The core argument for maintaining an automated build is that publishing bugs are mostly avoidable.

Answer 3

Put the estimate in hours: Estimate how much time you currently spend, and how much time you'd spend

Put the estimate in customer complaints: Estimate your current number of bugs. Estimate how many of those bugs the new system would have caught. Find out the percentage of bugs reported by users, and document how many fewer bugs will be user visible.

Add to the hours: Figure out how long it would take to fix the bugs that would be caught, and tack that onto the hourly estimate.

Add a non-quantifiable "salability".
With the extra time, we build extra features. With fewer bugs, that's fewer demos where sales guys shoot themselves in the foot. How many extra copies of software can we sell if we do this?

The last bit won't be successful; it's there primarily to focus attention on the first two metrics; hours and customer-visible defects.

Answer 4

My advice is to discredit what's there now then offer the alternative.

You could try putting the emphasis on the problems of how you do build and deployment now and win that battle first. Manager's don't want to change something that isn't causing them grief so you need to prove it is going to be a problem if nothing's done.

You should consider: how much time and credibility is lost with bad builds; how many mistakes are made currently; how much manual repetition takes place etc. and try to put metrics or examples of these things.

If you can win the support of your developers then you can also add their approval to the strength of your argument. Another point to make is that good developers like to work with good tools so progressive Management equals motivated developers.

If you win the hearts-and-minds of the developers and your Manager that may mean more than a piece of paper with some numbers on it.