Keep your Source Close and your Unit Tests Closer

Question

When I first started using unit tests I encountered two problems. First was being able to test private methods and fields and second falling behind on keeping unit tests up to date when rapid development was taking place. Consequently I adopted the approach below for my unit tests.

#if UNITTEST
using NUnit.Framework;
#endif

public class MyBlackMagic
{
   private int DoMagic()
   {
      return 1;
   }

   #if UNITTEST

   [TestFixture]
   public class MyBlackMagicUnitTest
   {
        [TestFixtureSetUp]
        public void Init()
        {
             log4net.Config.BasicConfigurator.Configure();
        }

        [Test]
        public void DoMagicTest()
        {
             Console.WriteLine(System.Reflection.MethodBase.GetCurrentMethod().Name);
             Assert.IsTrue(DoMagic() == 1, "You are not a real magician!");
         }
     }

     #endif
 }

I find that approach overcomes my two problems and it's a flick of a pre-compiler switch to make sure all the unit tests compile.

My problem now is that I am moving to a new project where the talk is of using seperate assemblies to hold the unit tests. Before I dive in and start expounding on the virtues of the internal class approach as shown above I'd like to know if anyone thinks it has any shortcomings?

Edit:

Just to add a couple of points around some of the mentioned weaknesses:

• The unit testing code will never impact the production code since the UNITTEST pre-compiler flag gets switched off,
• The unit test code doesn't make the main code any less readable since it's placed at the bottom of each class and wrapped in a Visual Studio region directive,
• I find the internal unit test class means the main class is actually simpler as there are no extra methods or properties that have to be exposed just for testing. There will always be cases where you want to test some internal state of a class as part of a unit test sooner or later...

Answer 1

I think that's not what unit tests were designed for. If you have developed code quickly and your unit tests are behind, then they will fail of course, but this shouldn't lead you to use black magic, but to write the unit tests. If you don't like this way, then you shouldn't use unit tests at all.

Answer 2

I find this approach very ugly, since it clutters your real logic with testing methods, making your source harder to read.

Next to that, you also have a dependency (reference) to the NUnit assemblies in your project itself. Although the dependency is not necessary when you compile without the unit_test conditional define, this is plain ugly and unneccessary.

If you want to keep up with your unit-tests, I advise you to write tests first, and then implement the real code.

Writing unit-tests is more then just about testing; it is also about designing code.

By writing the test first, you'll get to think of the API / the interface of your class, or how you want to use those classes.

Answer 3

If you can't keep all of the tests succeeding all of the time (because of development deadlines) then I think your not taking unit testing very serious and you should take the maintenance of the tests into account when making your estimates.

Answer 4

You shouldn't test private methods separately, because they are (should be!) used from public methods or possibly constructors - so the public methods depends on the private ones to succeed with their assignments. These public methods/constructors will (should!) fail if the private methods don't do their work. So your approach is actually a bad approach for writing unit tests. And to iterate the previous answers - write your unit tests before you write your methods.

Answer 5

Try changing your private methods to internal methods and using the InternalsVisibleTo attribute.

EDIT: I should qualify that by saying that I personally don't think having the TestFixture classess in the same assembly as the classes being tested is a good way to go. I've never had any problems with keeping my unit tests in a seperate assembly. My team has adopted a naming conventions of suffixing the unit tests assembly with ".UnitTests"

The InternalsVisibleTo attribute allows me to fully test non-public methods within my classes, as well as testing public methods that may use them.

Answer 6

I find that keeping my unit tests in their own assembly works very well, and haven't run into any problems using it.

However, if you find yourself needing to test private members of some class, that probably means your class is doing a lot of stuff. You may be better off extracting those private members into a new class. Then you can test that new class directly via its public methods.

Answer 7

The rationale behind keeping test code in it's own seperate assembly is because test code is not supposed to be "used" by the user. Unit tests are only there to verify the code does what it is specified to do. The common practice is to put test code in it's own assembly so that the production code is not dependent on the test code and the unit test framework.

From this code example, it's not clear to me what you're trying to test. It seems like you're trying to test something that isn't in completely isolated from the environment. Why would you need a logger when xUnit test runners can log test results for you?

Internal classes are tricky to test because you need a public interface (or abstract class) that the internal class is implementing and you need to test that together with the class that created it. More explicitly unit tests should check the behavior of one specific class, and if an internal class is returned in a way, then you need to check it's returned in a correct manner.

Answer 8

And how, pray tell, do you run your unit tests on any build for which you've set UNITTEST to false?

I'm a crappy, lazy implementer of TDD. I don't write anywhere near enough tests, and I don't run them way too infrequently. But even I test my release builds.