views:

218

answers:

3

Hello there,

I have solution with >270 projects, it contains various folders and etc. Imagine each project have Unit Tests, what do you think what is the best way to organize them? Should each project have Unit Tests near by, or I should create special folder for them, or even different solution for just unit tests.

How do you organize them for such a big solutions/projects ?

+4  A: 

A single solution with 270 projects sounds like a problem to start with. How long does it take to open VS? :) (Seriously, can you either combine some projects together or split the solution?)

Typically I keep the unit tests in their own project, but in the same solution. Within the project, mirror the folder/namespace structure of the production project. Have a look at how Noda Time is organised for an example. I've been at companies where they've been kept in a separate solution, but that was really painful. (They had a good-ish reason: the tests were in VS2005 and the production code was 2003.)

Sometimes I've set the default namespace of the test project to be the same as the production project - that's more important in Java than in .NET (as it lets you get at package access members) but it can still be helpful as it means the class you're going to test doesn't need to be imported.

Jon Skeet
1 unit test project is the way to go. We tried to mirror our projects and create test project for each - it ended up with broken tests all around.
Arnis L.
I don't like having them in a separate project because it makes it more difficult to test internal classes and methods.
tster
@Arnis: Why? I've never found any problem with one test project per production project. What problems did you run into?
Jon Skeet
@tster: That's what InternalsVisibleToAttribute is for. See http://msdn.microsoft.com/en-us/library/system.runtime.compilerservices.internalsvisibletoattribute.aspx - whereas with the test code in the production project, you end up with references to test frameworks etc from your production assembly, as well as having all that test code (and potentially significant amounts of test data) in the assembly too. Ick.
Jon Skeet
@Jon, I freakin' love this sight!!! I learn new stuff every day. This is great!
tster
@tster: One very important rule of unit testing: Only test the public API (protected counts as public). Testing internals will inhibit refactoring, and in any case you should be able to hit all code through the public API.
Mark Seemann
@Amis L.: I strongly disagree - see my answer for further reasons.
Mark Seemann
@Jon: It takes some time to open VS.. (2-5 mins.):) But I can't split it now.. what can I do, it's huge thing. I will start new version of it soon, so It hopefully will be beautiful and not so huge :)
Lukas Šalkauskas
@Mark: I disagree. There are plenty of cases where you expose a very high level API which is really hard to unit test, but perfect for the consumer - but the internal API is easy to test. If you restrict yourself to the public API you end up writing acceptance tests more than unit tests. Yes, the cost of refactoring does go up a bit - but I think it leads to better design of the internal API, more robustness, and simpler testing in general.
Jon Skeet
A: 

After switching to Gallio from the built in Visual Studio testing framework I have started putting my tests in a Test folder within the same project. For little helper classes I might even define a unit test right there in the same folder. As long as your namespaces are relatively small (I don't like to have sprawling namespace folders) this doesn't add too much clutter for me. Anyways, later on, when those classes have been around for a while and are stable, you can move those unit tests to a separate folder.

tster
So when you ship your production binaries, do they still have the test classes in? Do you ship the test framework assemblies as well, which presumably your project has references to? What about test data?
Jon Skeet
I've never thought about that too much because I work on internal apps. But yes, I suppose the test code would be shipped. I'll have to consider this in the future.
tster
+1  A: 

You should have one (or more) unit test projects per target project. This is the only way you can stay flexible and vary each unit test suite together with the target project.

If you have one unit test project covering more than one target project, this creates an artificial tight coupling between these two projects, because you will not be able to compile the unit test project without all of its target projects. This, again, makes it really hard to test a single project in isolation - which is what unit testing is all about.

If you need to share test code between several test targets, you can always create a shared library with a generalized, reusable test API, as long as it doesn't reference any of the target projects.

That said, I have to agree with Jon Skeet that a solution with 270 projects is a structural smell that must be addressed first (unless it is a "build all" solution used for automated builds).

Mark Seemann