Is there hard information on the relative productivity of top programmers?

Question

It seems to be an article of faith that top-flight programmers are several times as productive as mediocre programmers. Can anyone point to studies or reports which support this with hard data?

Answer 1

It all starts with this article (I'm pointing you to the citations thereof, wherein you will find 40 years' worth of discussion and further "hard data" studies).

Answer 2

There are some statistics quoted in "Code Complete". The studies showed a factor of 20:1 in productivity.

Answer 3

I would be very suspicious of any such studies, especially when it comes from large corporations.

There is no objective or even professionally relative benchmark for how long task X should take. At least a civil engineer can say building this x mile road should take Z hours.

The hardest, but most important part of the job is time management. I hate picking target dates, because unexpected stuff always comes up.

Then there is the question of leadership in the team. A lot of programmers surf all day because the team lead doesn't distribute work and manage people well.

It also sucks as a good programmer to feel unrecognized for performing a nigh-miracle and no one understands or appreciates what you just did and how helpful it was. This happens a lot to debuggers/firefighters.

There isn't a simple answer to the question, because the problem isn't simple.

Answer 4

It's hard to find hard data on anything related to programming, but I'll second the suggestion by Shiraz Bhaiji that you look at Code Complete by Steve McConnell. You might want to look at the next book he wrote, Rapid Development, as well. It also has some statistics.

As comments above pointed out, you need a good definition of productivity. Some people say that experienced and inexperienced programmers both write about the same number of lines of code per day but experienced programmers keep more lines of code per day. The inexperienced programmers will spin their wheels and rewrite most of today's code tomorrow.

Answer 5

There's some reference on this point in The Mythical Man-Month; I don't have my copy with me right now to see if the information is well-sourced, though.

Answer 6

The Pareto Principle (also known as The 80/20 Principle) lies behind a lot of non-intuitive statistical outcomes throughout Life, The Universe and Everything.

In it's crudest form, approx 80% of the outputs from a system can be accounted for by approx 20% of the inputs.

Applying it to this case, approx 80% of the productivity comes from approx 20% of the developers.

Yes, this isn't a hard study but I think it accounts for some of the inner reasoning behind many managers who assume the principle applies to software development...which, just through personal observation, I think it probably does.

Richard Koch is the name to Google for if you want to find out more about this natural phenomenon.

Answer 7

Actually, your absolute best developers will often be less productive by dumb measures, as they spend a lot of their time helping the other developers out of jams.

Answer 8

Here's my hard and fast method:

• Start with 168 hours in the average week
• Subtract the number of hours spent on Stack Overflow
• Subtract the number of hours twittering with your fellow twitterati
• Subtract the number of hours spent sleeping
• Subtract the number of hours spent in the loo
• Subtract the number of hours spent doing programmer quizzes like code golf and ToughestDeveloperPuzzleEver
• Subtract the number of hours trying to decide what to have for lunch/dinner
• Subtract the number of hours actually spent eating and making smalltalk with coworkers/family
• Subtract the number of hours procrastinating about work you should be doing
• Subtract the number of hours spent in meetings

If somehow after all that, you're not in negative figures, then you're doing better than the 90% of all other programmers :P

Answer 9

Yes.

In Peopleware, Tom DeMarco documented a study he has done in productivity via experimental "Code Games" over several years with a reasonable sample size.

On individual performance he found;

1. The best people out performed the worst by 10:1
2. The best people out performed the median by 2.5:1
3. The top half above the median outperformed the bottom half by 2:1

He found there were also a bunch of unrelated factors, such as salary, language used and years of experience. Interestingly he found "... the best performers are clustering in some organisations while the worst performers are clustering in others". This means; look at the people around you, you probably perform closer to them than others in better or worse organisations.

It's an interesting experiment conducted over several years and on the whole a book well worth reading.

See p44 - 48, Peopleware 2nd ed, Tom Demarco.

Answer 10

This Joel Spolsky's article contains what can be described as "hard data". One interesting observation of the article is that quality of work and time are uncorrelated in the data presented.