Do you put unit tests in same project or another project?

Question

Do you put unit tests in the same project for convenience or do you put them in a separate assembly?

If you put them in a separate assembly like we do, we end up with a number of extra projects in the solution. It's great for unit testing while coding but how do you release the application without all of these extra assemblies?

Answer 1

In my opinion, unit tests should be placed in a separate assembly from production code. Here are just a few cons of placing unit tests in the same assembly or assemblies as production code are:

1. Unit tests get shipped with production code. The only thing shipped with product code is production code.
2. Assemblies will be unnecessarily bloated by unit tests.
3. Unit tests can affect build processes like automated or continuous build.

I don't really know of any pros. Having an extra project (or 10) isn't a con.

Edit: More Info On Build and Shipping

I would further recommend that any automated build process place production and unit tests into different locations. Ideally, the unit test build process only runs if the production code builds, and copies the product files into the unit tests directory. Doing it this way results in the actual bits being separated for shipping, etc. Additionally, it is fairly trivial to run automated unit testing at this point on all tests in a particular directory.

To summarize, here is the general idea for a daily build and testing and shipping of bits and other files:

1. Production build runs, placing production files into a specific "production" directory.
   1. Build production projects only.
   2. Copy compiled bits and other files into a "production" directory.
   3. Copy bits and other files into a release candidate directory, aka a Christmas release directory would be "Release20081225".
2. If production build succeeds, unit test build runs.
   1. Copy production code to "tests" directory.
   2. Build unit tests to "tests" directory.
   3. Run unit tests.
3. Send build notifications and unit tests results to developers.
4. When a release candidate (like Release20081225) is accepted, ship these bits.

Answer 2

My unit tests always go in a separate project. In fact, for every project I have in my solution, there is a separate test project that goes along with it. Testing code is not application code and should not be intermingled with it. One advantage to keeping them in separate projects -- at least using TestDriven.Net -- is that I can right-click on a test project and run all the tests in that project, testing an entire library of application code with one click.

Answer 3

Separate project, but in the same solution. (I've worked on products with separate solutions for test and production code - it's horrible. You're always switching between the two.)

The reasons for separate projects are as stated by others. Note that if you're using data-driven tests, you might end up with quite a significant amount of bloat if you include the tests in the production assembly.

If you need access to the internal members of the production code, use InternalsVisibleTo.

Answer 4

I fluctuate between same project and different projects.

If you're releasing a library releasing the test code with the production code is a problem, otherwise I find it usually isn't (although there's a strong psychological barrier before you try).

When putting tests in the same project I find it easier to switch between tests and the code they test, and easier to refactor/move them around.

Answer 5

I put them in separate projects. The name of the assembly mirrors that of the namespaces, as a general rule for us. So if there is a project called Company.Product.Feature.sln, it has an output (assembly name) of Company.Product.Feature.dll. The test project is Company.Product.Feature.Tests.sln, yielding Company.Product.Feature.Tests.dll.

You are best keeping them in a single solution and controlling the output via the Configuration Manager. We have a named configuration for each of the main branches (Development, Integration, Production) in lieu of using the default Debug and Release. Once you have your configurations setup, you can then include or exclude them by clicking on the "Build" checkbox in the Configuration Manager. (To get the Configuration Manager, right-click the solution and go to Configuration Manager.) Note, that I find the CM in Visual Studio to be buggy at times. A couple of times, I have had to go into the project and/or solution files to clean up the targets that it created.

Additionally, if you are using Team Build (and I am sure that other .NET build tools are the same) you can then associate the build with a named configuration. This means that if you don't build your unit tests for your "Production" build, for example, the build project can be aware of this setting as well and not build them since they were marked as such.

Also, we used to do XCopy drops off of the build machine. The script would just omit copying anything named *.Tests.Dll from being deployed. It was simple, but worked.

Answer 6

I do not understand the frequent objection to deploying tests with production code. I led a team at a small microcap (grew from 14 to 130 people). We had a half-dozen or so Java apps and we found it EXTREMELY valueable to deploy tests into the field to execute them on a specific machine that was exhibiting unusual behavior. Random problems occur in the field and being able to throw a few thousand unit tests at the mystery with zero cost was invaluable and often diagnosed problems in minutes...including installation problems, flaky RAM problems, machine-specific problems, flaky network problems, etc, etc. I think it is incredibly valuable to put tests into the field. Also, random problems pop up at random times and it is nice to have the unit tests sitting there already waiting to be executed at a moments notice. Hard-drive space is cheap. Just like we try to keep data and functions together (OO design), I think there is something fundamentally valuable in keeping code and tests together (function + tests that validate the functions).

I would like to put my tests in the same project in C#/.NET/Visual Studio 2008, but I still haven't investigated this enought to achieve it.

One big benefit of keeping Foo.cs in the same project as FooTest.cs is that developers are constantly reminded when a class is missing a sibling test! This encourages better test-driven coding practices...holes are more apparent.

Answer 7

Separate projects, although I debate with myself whether they should share the same svn. At the moment, I'm giving them separate svn repositories, one called

"MyProject" - for the project itself

and one called

"MyProjectTests" - for the tests associated with MyProject.

This is fairly clean and has the advantage that commits to the project and commits to the tests are quite separate. It also means you can hand over the project's svn if needed, without having to release your tests. It also means you can have branch/trunk/tag directories for your tests and for your project.

But I'm increasingly inclined to have something like the following, in a single svn repository, for each project.

MyProject
|\Trunk
| |\Code
|  \Tests
|\Tags
| |\0.1
| | |\Code
| |  \Tests
|  \0.2
|   |\Code
|    \Tests
\Branches
  \MyFork
   |\Code
    \Test

I'd be interested to know what other people think of this solution.