What are the advantages of self-testing code vs separated tests?

Question

Personally, I've always put unit tests in a separate project just because that's how MSTest seems to be set up. But I'm reading Refactoring: Improving the Design of Existing Code by Martin Fowler and he seems to be advocating not only putting them in the same project, but also putting them in the same class as the method they're testing.

I'm really honestly having trouble thinking of ways this is different from having the tests in a separate area from the code itself other than philosophical differences (are tests documentation or clutter?).

Are there any clearcut reasons to choose one over another? Or is this mainly a philosophical difference?

UPDATE: I'm not necessarily convinced yet one way or another, but at least I have an idea what the arguments are. I wish I could select everybody's answer, but I had to select just one.

Answer 1

It's mainly a philosophical difference I think.

There may be marginal performance advantages in having them in a separate project (in so much as your tests aren't deployed into production), but they probably are not that significant.

Also remember that Refactoring was written quite a while ago (in IT terms) so preferred practices may well have moved on since then.

Answer 2

Putting them in the same class being tested may break popular unit testing frameworks, as NUnit won't test types that don't have a default parameterless constructor.

Putting the tests in a different file but same project is better, but still causes your main project to reference testing frameworks like NUnit.Framework.dll as well as any mocking frameworks like Rhino.Mocks.dll.

Having your tests inside the class under test also increases the size of your distributed project.

Separate the tests out into a separate project, and you don't have any of these issues.

Answer 3

Perhaps there is some elegance to having self-testing code, but I tend to side with the same philosophy as you -- that code separation trumps some notion of abstract beauty. When you design a class, you can fundamentally break it into three parts:

• What the class does (eg, the class' definition)
• How it does it (the implementation)
• How you use it (documentation and/or test cases)

I see test cases as serving the purpose of documentation, as well as part of a safety net in a test suite. When a new programmer is looking at your code, possibly long after you've stopped working on it, the documentation is seldom the most effective way of communicating how the class should be used. It can answer questions about how the code behaves in specific situations, providing a general overview of the class and it's methods, and so on, but the test cases provide a concrete example of how the class would be used in real code.

So for that reason, I would tend to say that they should remain outside of the class itself, since this re-enforces this degree of separation.

Answer 4

Keeping your tests in a separate area (subdirectory, project, whatever) is a good way to retain the option to deploy just the production code, which is usually what you'll want to do.

Answer 5

There's a lot of convenience in having your tests in the same class that you are testing. Quite often (and prior to embarking on TDD), I used add a Main method to the class just to test the functionality of the code I wrote. And I didn't remove the main method because it was quite useful to have the test code lying around. This was convenient, in the same sense that it is convenient to have documentation tied to your code. I didn't have to look else where or search for the test code.

Having said that, the point of TDD is not just about testing/quality-assurance. It's more about getting you to think about your design/interfaces. And from that view point, it makes sense to have your "test" (a.k.a "design") code in a separate file. Your "test" code is the client/user of your class.

If the purpose of writing tests in a separate file were a technical one, i.e "because the unit test framework was designed that way" or "otherwise you'll need to include a reference in your main project", or "it improves performance", that doesn't sound very convincing. If that were the case things would have been done very differently.

Answer 6

I don't see any benefits in having test code and production code in the same class. Instead I see some disadvantages:

• It won't be possible to deploy and distribute the production code without the test code. So instead of a 100KB file you might need to ship a 200KB or larger file. (When using TDD, there lines of test code is often equal or greater to the lines of production code.)

• The structure of the tests is too tightly coupled with the production code. There should be no 1:1 relation between the tests and the production classes. Instead, there should be a 1:1 relation between tests and behaviour.

Quoted from http://blog.daveastels.com/files/BDD_Intro.pdf

"When you realize that it's all about specifying behaviour and not writing tests, your point of view shifts. Suddenly the idea of having a Test class for each of your production classes is ridiculously limiting. And the thought of testing each of your methods with its own test method (in a 1-1 relationship) will be laughable."

I program mainly in Java and I use Maven to build my projects. There I have the tests in the same module and package as the production classes that they operate on, but in a different directory (/src/main/java and /src/test/java). When Maven builds the project, it executes all tests, but only the production code is included in the binary distributable.