C# - Excluding unit tests from the release version of your project

Question

How do you usually go about separating your codebase and associated unit tests? I know people who create a separate project for unit tests, which I personally find confusing and difficult to maintain. On the other hand, if you mix up code and its tests in a single project, you end up with binaries related to your unit test framework (be it NUnit, MbUnit or whatever else) and your own binaries side by side.

This is fine for debugging, but once I build a release version, I really do not want my code to reference the unit testing framework any more.

One solution I found is to enclose all your unit tests within #if DEBUG -- #endif directives: when no code references an unit testing assembly, the compiler is clever enough to omit the reference in the compiled code.

Are there any other (possibly more comfortable) options to achieve a similar goal?

Answer 1

I definitely advocate separating your tests out to a separate project. It's the only way to go in my opinion.

Yes, as Gary says, it also forces you to test behavior through public methods rather than playing about with the innards of your classes

Answer 2

I've always keep my unit tests in a seperate project so it compiles to it's own assembly.

Answer 3

For each project there is a corresponding .Test project that contains tests on it.

E.g. for the assembly called, say "Acme.BillingSystem.Utils", there would be a test assembly called "Acme.BillingSystem.Utils.Test".

Exclude it from the shipping version of your product by not shipping that dll.

Answer 4

I would recommend a separate project for unit tests (and yet more projects for integration tests, functional tests etc.). I have tried mixing code and tests in the same project and found it much less maintainable than separating them into separate projects.

Maintaining parallel namespaces and using a sensible naming convention for tests (eg. MyClass and MyClassTest) will help you keeping the codebase maintainable.

Answer 5

The .Net framework after v2 has a useful feature where you can mark an assembly with the InternalsVisibleTo attribute that allows the assembly to be accessed by another.
A sort of assembly tunnelling feature.

Answer 6

As the others point out, a seperate test project (for each normal project) is a good way to do it. I usually mirror the namespaces and create a test class for each normal class with 'test' appended to the name. This is supported directly in the IDE if you have Visual Studio Team System which can automatically generate test classes and methods in another project.

One thing to remember if you want to test classes and methods with the 'internal' accessor is to add the following line to the AssemblyInfo.cs file for each project to be tested:

[assembly: InternalsVisibleTo("UnitTestProjectName")]

Answer 7

I definitely agree with everyone else that you should separate the tests from your production code. If you insist on not, however, you should define a conditional comiplation constant called TEST, and wrap all of your unit test classes with a

#if TEST
#endif

first to ensure that the test code does not compile in a production scenario. Once that is done, you should either be able to exclude the test dlls from your production deployment, or even better (but higher maintenance), create an NAnt or MSBuild for production that compiles without the references to the test dlls.

Answer 8

As long as your tests are in a seperate project, the tests can reference the codebase, but the codebase never has to reference the tests. I have to ask, what's confusing about maintaining two projects? You can keep them in the same solution for organization.

The complicated part, of course, is when the business has 55 projects in the solution and 60% of them are tests. Count yourself lucky.

Answer 9

I always create a separate Acme.Stuff.Test project that is compiled separately.

The converse argument is: Why do you want to take the tests out? Why not deliver the test? If you deliver the tests along with a test runner you have some level of acceptance test and self test delivered with the product.

I've heard this argument a few times and thought about it but I personally still keep tests in a separate project.

Answer 10

I put the tests in a separate project but in the same solution. Granted, in big solutions there might be a lot of projects but the solution explorer is good enough on separating them and if you give everything reasonable names I don't really think it's an issue.

Answer 11

Yet another alternative to using compiler directives within a file or creating a separate project is merely to create additional .cs files in your project.

With some magic in the project file itself, you can dictate that:

1. nunit.framework DLLs are only referenced in a debug build, and
2. your test files are only included in debug builds

Example .csproj excerpt:

<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="3.5">

   ...

   <Reference Include="nunit.framework" Condition=" '$(Configuration)'=='Debug' ">
      <SpecificVersion>False</SpecificVersion>
      <HintPath>..\..\debug\nunit.framework.dll</HintPath>
   </Reference>

   ...

   <Compile Include="Test\ClassTest.cs" Condition=" '$(Configuration)'=='Debug' " />

   ...
</Project>

Answer 12

One thing yet to be considered is versions of VisualStudio prior to 2005 did not allow EXE assembly projects to be referenced from other projects. So if you are working on a legacy project in VS.NET your options would be:

• Put unit tests in the same project and use conditional compilation to exclude them from release builds.
• Move everything to dll assemblies so your exe is just an entry point.
• Circumvent the IDE by hacking the project file in a text editor.

Of the three conditional compilation is the least error prone.

Answer 13

If the #if(DEBUG) tag allows for a clean "release" version, why would you need a separate project for tests. The nunit LibarryA/B example (yeah, I know its a example) does this. Currently wrestling with the scenario. Had been using a separate project, but this seems to possibly allow for some productivity improvements. Still hummin and hawin.