Has any research been done to scientifically evaluate various coding standards?

Question

As we all know, in the software development world there is a huge amount of passionate debate about coding standards: brackets on the same line or new line, when to indent and by how much, tabs or spaces, etc.

Has anyone ever done any actual scientific research to determine what coding standards produce the "best" code (where "best" could be any specific criterion: readable, error-free, etc)?

How would one even go about doing this? How does one quantify what is the "best" standard? Clearly it would have to be language-specific. Is it even possible?

Answer 1

To do this even remotely "scientifically", one would have to set up two or more teams to implement the same (realistically sized) solution but using different coding standards and then compare the results. The costs of doing this would be huge, so I suspect the answer to your question is - no.

Answer 2

The only research result I've seen is that code in all-caps is harder to read than code in either all-lowercase or mixed case. (Sorry, can't give a reference.)

Answer 3

Why would anyone need science when its so clear the the coding style I use is the "One True Style"? :)

Answer 4

I remember one research that measured the effect of indentation. I think it was referenced in Code Complete 2, but I don't have it at hand, so I try to quote from memory:

Both senior and beginner programmers were given the task of determining errors in code examples - indentation in these examples varied from 0 to 8 spaces.

The result was, that when code was unindented, the difference between beginners and seniors was quite small, but when the code was indented, the seniors were a whole lot better. While for junior programmers the indentation didn't matter much.

It was also determined that the amount of indenting was not that important as long as it's anywhere from 2 to 6 spaces. Both below and above that was considerably worse.

This research also highlights an important thing in coding standards - programmers are most effective when they work with a source code that's coded using an already familiar standard.