From a Management POV: How much is a code review worth?

Question

It's obvious to me that code reviews (peer reviews) improve the quality of the generated code.

I've worked hand to hand in some code with some of my coworkers and, specially with some of them, code was cleaner and a lot better. Reviewing other people code is not the same thing. You're not familiar with the code, you haven't made the vast majority of the design decisions and it can be quite frustrating until you really understand what's going on... even for the best written (non trivial) code.

I tend to always say that coding, no matter how fun it is, is part of a business for money so my question is: Is it really worth it? I would have two developers just to get a level of excellence that might as well be achieved by a little more time with just one of them. Have you found it useful anytime?

Note: Altough I'm mostly a developer, I ask this from the manager point of view, from the "resource" (as in you're all just numbers) point of view :P

Answer 1

Bugs are expensive. Code reviews decrease the probability that bugs exist.

Code reviews are worth a fair bit of money. An actual number on this is very difficult to come by. Code reviews, when done properly, catch bugs, errors, bad tests, and potential legal issues in the code.

The potential legal issues may be: unattributed/unlicensed code, mixing gpl code with non-gpl code (assuming that the final product will not make the source available).

The issue is not how they cost, but how much loss are they protecting against. Additionally if code related problems (errors, bugs, bad tests) are fix, much trouble and effort will be avoided in later releases.

To get the most out of this, treat this as an optimization problem where you can adjust the amount of of code review to optimize the best product. You may not be able to put much to this, but later assuming these conditions:

• The code reviews are successful many bugs are found and fixed
• Customer likes the changes

You should receive more profit, and would be able to attribute more developers to the product and more resources to code reviews.

Answer 2

I guess, they worth it, when your management cares about the quality of the product, not just the speed of getting out to the market (which could be a valid scenario though: "Just ship the damn thing!").

Answer 3

Short term, maybe for management they aren't. In the long run, they clearly do - and have tangible benefits. Less bugs, better extensibility of code, easier to maintain etc.

Some managers are simply short-sighted, and need to be educated of the benefits of code reviews.

Answer 4

It depends on how code reviews are done, as to whether they are worth it, and what else is in the system.

For example, if you don't have any unit tests, each person designs their own classes, and a manager does the review in a locked office, then, it would be of limited use.

But, if you have unit tests, and reviews are done by the developer in front of peers, then it can be very helpful, as he gets feedback from others, and everyone can learn from the experience.

So, I think the best way to answer this is, it depends on the development system you have in place.

Answer 5

In addition to the reduction of bugs, you also have the side-benefit that other developers become familiar with a given portion of the code. That promotes code reuse, for one, but a potentially more important benefit is having other developers ready to step in and make changes to that module if the original developer leaves the company (or even is on vacation when a critical bug is found).

Answer 6

The question is: do you want to remove bugs from your software? If so, code reviews are a very cost-effective way to do this.

Answer 7

This is a link to an interesting interview of the follow stackoverflower Alex Martelli about Code Review.

Edit: I am going to mark this answer as community wiki since credit should actually go to Alex. It is more about code review in the open source community, but I think he also makes some good points which are relevant for businesses.

Answer 8

Code quality is one other thing to consider along with the other answers on this post.

Code review helps maintaining a quality code base where people cannot just shove in bad code into the code base. A bad code base can come back to hunt you and can have serious impacts on your costs. It could be a burden.

If there are members on the team that don't properly understand the design of the applications they're writing code against, you might end up with changes that break the design. This could result in a bad code base that might be more difficult to change later on (already released, dependencies, etc). It could also limit the extensibility of your applications and result in a slow reaction to changing requirements. This can be very expensive, and if you already have a lot of such code deployed out there also difficult to achieve.

Having a code review done by peers will diminish the possibility of inappropriate code ending up in the code base, which will generate a better quality of the code base.

I think it's beneficial to have a code review practice, but also have some guidelines as to what to look for when making a review.

Answer 9

Code reviews do help to keep code clean, to catch bugs earlier (which just can't be wrong) and to share knowledge and best practices (this is another very important benefit). In other words, code reviews help to keep the technical debt under control and to amplify learning, and both are required to achieve high quality when a team includes less experimented programmers.

And without high quality, you'll loose time (and thus money) on the long run.

So, good code reviews definitely aren't waste, they will actually help you to save money. Bug are waste, certainly not high quality. High quality is a key to high productivity and sustainable pace.

(EDIT: For some data on the ROI of code review, I'd suggest to check the sample chapter The Case for Peer Review from the Best Kept Secrets of Peer Code Review book from SmartBear, especially page 9. As a manager managing resources (sigh), you might find it interesting and it might change your point of view on what is really waste.)

Answer 10

There is plenty of evidence to support code reviews:

• The classic Fagan study
• A case study where the company would have saved 200 thousand dollars had they been doing reviews
• There are more studies listed in this wikipedia article
• And the Software Inspection book contains some more case studies.

There are many more case studies that you can find online.

Speaking from personal experience, I've only briefly worked without code reviews and the code I wrote during that time was some of the worst I've ever written.

But if it doesn't work for you, don't do it. I think the evidence suggests that you should at least give it a fair shot, measure the results for yourself and make a decision based on those results.

Answer 11

To my mind there is also the question of how are code reviews valued with in the culture of the company. If they are viewed as a bureaucratic requirement that isn't worth diddly, then perhaps they aren't a worthwhile exercise. On the other hand, there can be the view that reviews help make for a more homogeneous code base and spread knowledge of various bits of functionality. If the culture supports looking over what was done and where it can be improved, that would support doing them.

Alternatively, developers could go rogue and just do this on their own without management knowing anything about it, too.

Answer 12

"You're not familiar with the code, you haven't made the vast majority of the design decisions and it can be quite frustrating until you really understand what's going on... even for the best written (non trivial) code."

You say "best" written, but one of the things that code reviews teaches you is to write code that other people can understand more easily. Because if you don't write code that's easy to understand, your reviewer will be frustrated, and while you're not looking will put sugar in your gas tank and change your keyboard to Dvorak. Code should take much less time to review than it does to design and write - if it doesn't, then the presentation of the code is sub-standard. So I don't think you should consider code review to take two developers, it really should only be "one and a bit".

Of course reading code on your own will never be quite as easy as with the author there to explain it. But if people unfamiliar with the code (this includes the original author, a year later) are to bugfix, refactor, or add new functionality, then the code has to be readable by people who don't understand it. Learning to write legible code takes time, but applying what you've learned really isn't that costly - clear code doesn't take much longer to write than obscure code. So being forced to write code that other people can read quickly is an excellent investment, IMO.

In terms of a second POV catching errors, co-operating during programming offers similar benefits to code review, quite possibly greater. It doesn't necessarily provide the same incentives for the code to be self-explanatory. So I think everyone benefits from having their code reviewed, at least sometimes, even if it's not your main means of ensuring correctness.

Look at it another way - you say your colleagues currently write better, cleaner code when you're looking over their shoulder, but not when you're not. Reviewing their code after the fact, and giving them good feedback, is like looking over their shoulder when you aren't even in the building. And the same from them to you - I think 90% of the clarity of my own code (such as it is), comes from me looking at what I've written and thinking, "when people read this, what will they think?". I've seen the code I wrote back when I was a solo amateur programmer, and it was impenetrable. Now, even when it's a personal project, and "people" is just me, there's a certain frame of mind you get from imagining you have readers.

Anyone who's ever worked with my code, now's your chance to say that it's still impenetrable ;-)

Answer 13

I give you a real-world analogy to explain why code reviews actually save a lot of money:

Imagine a street with several street lamps, some of which are faulty. It's obviously cheaper to check and repair (if necessary) all of them, instead of waiting for the first one to fail, go there, repair it, go back, wait for the next one etc.

Same with code: When newly written, it most likely contains several bugs. It's cheaper to find many of them in a code review than to wait for the first bug to pop up (hopefully during tests), have a developer fix just that one (which also takes some time reading into that part of code), wait for the next one to pop up etc.

Answer 14

"Invaluable" is the short answer you're looking for.

You're not "wasting" (as you put it in one comment) one developer by having him or her look over another developer's code. By having a strict code review regimen you are going to increase the quality of your code, and by extension, the quality and reliability of your product. The expense you incur up front by taking the time to review the code will be more than offset by the time saved fixing bugs downstream.

There's an axiom that says a bug costs 10 times as much to fix for every stage of development it gets past. That is, a bug that makes it to QA costs 10 times as much to fix as one that was caught by development in the first place. A bug that makes it past QA costs 10 times as much to fix as one that was caught by QA, and 100 times as much to fix as one that was detected early on by development in a code review, automated unit test, or whatever. I think there's a lot of truth in this particular theory.

Moreover, you suggest that

Reviewing other people code is not the same thing. You're not familiar with the code, you haven't made the vast majority of the design decisions and it can be quite frustrating until you really understand what's going on... even for the best written (non trivial) code.

This isn't such a big problem if you're doing regular design and code reviews. The very act of participating in these reviews makes one by definition "familiar with the code," and addresses the issue that the reviewer hasn't been involved in the majority of design decisions (because they have).

Answer 15

An additional point to all of the above is that without a code review or some other systematic way of having a public discussion of code and preferred practice you run the risk of creating a culture where quality doesn't matter.

I've worked in a number of places where there was no mechanism like code review in place, and it generally led to a balkanization of the codebase. Even the most well intentioned developers built things based on their own preference. Not only do you get a mess of different standards, but developers tend to work as if they're working on a solo project - everything works the way they're used to working, so they can't see a problem.

Answer 16

Our clients consider code review so important that for the largest clients we are contractually obligated to do code reviews.

I have never participated ina code review that I thought was a waste of my time. Either one or more bugs (or better ways to do something, not necessarily a bug) are found or at least one other person becomes familiar with the code. It's a win either way. I have learned new techniques from code reviews and I have taught new techniques in a code review. I have learned not only about the changes but the why behind them which helps later on when you are maintaining that code. Code reviews also tend to enforce standard ways of doing things which makes the code easier over time to maintain.

I forgot to tmention, all it takes is one code review which found one major, show stopper bug to justify the cost of pretty much all the code reviews that year.