Comparing development time

Question

For a project, I need to analyze the differences in development time between two different frameworks to implement the same thing.

I can't just time myself writing one then time myself writing the other, because I will almost inevitably write the second one faster having experienced pitfalls and fixes while writing the first one.

My first instinct would be to use simple metrics as a heuristic, such as number of bytes, number of word or LOC.

I know there exist other measurements of code size such as Halstead complexity measures, however I don't think these correlate accurately with development time.

In this particular instance, I'm interested in solely the development time. I'm aware that maintenance time probably does correlate fairly strongly with complexity measures, but assume this is for writing a completely new feature which has no dependencies on previously written code.

Suggestions?

Answer 1

I can't just time myself writing one then time myself writing the other, because I will almost inevitably write the second one faster having experienced pitfalls and fixes while writing the first one.

You can compensate for this by having two test subjects. Let test subject A code in framework 1 first, framework 2 second; let test subject B first use framework 2 then framework 1. Scale the values by the total development time of the developer to compensate for difference in skill, then average for each framework.

This is more of a science methodology answer than a programming answer though... :)

Answer 2

I suggest you apply the Goal-Question-Metric (GQM) approach:

1. Determine what your goal is.
2. Define one or more questions that will help you achieve that goal (once you obtain the answers).
3. Select the metrics that will allow you to answer those questions.

Good luck!