Unit test, NUnit or Visual studio ?

Question

Hi all,

I'm using Visual studio (sometimes resharper) to run my unit test.

I heard about NUnit, but I don't know many things about it...

Should I care about it ? Can it offer something better than visual studio ? Should I Use NUnit and why ?

Thanks for your help

Tim

Answer 1

NUnit is a unit testing framework, which is also supported by resharper. I think you are using Microsoft's unit testing framework, so NUnit is just an alternative to Microsoft's product ;)

Here is the link to the homepage of NUnit: http://nunit.org/index.php

Answer 2

NUnit has few advantages over MS-Test

1. Suite attribute - can aggregate tests and execute them separately (useful for large projects with fast and slow tests for example)
2. Readable Assert method, e.g. Assert.AreEqual(expected, actual) vs Assert.That(actual, Is.EqualTo(expected))
3. NUnit has frequent version updates - MS-Test has only one per VS version.
4. Many integrated runners including Resharper and TestDriven.NET
5. Expected exception message assertion - can be done using attribute in NUnit but must be done using Try-Catch in MS-Test

Answer 3

I am not sure of others but NUnit provides nice GUI and console to run your Unit Tests and also you can generate report of the result of NUnit Test execution which would give the detail of whethere the test has failed or passed and also what time did it take for your unit test

Answer 4

NUnit works with the Standard edition of VS.

Answer 5

From my current perspective (after 8 months of development with about 10 developers on average) I would advice against using MSTest for the following reasons

• The framework in itself is quite slow. I don't mean the test, they are as fast as you write them, but the framework in itself will almost always take more time to run your test suite, single tests etc.
• The need to keep a Test-Metadata file which always leads to complications when several developers are working on it (recreating e.g. the metadata etc.). Every other test suite doesn't need a metadata file. It is kind of nice to organize your tests but you can achieve the same through namespaces, classes and method names.
• Doing Continuous Integration, if you want to run unit tests on your build machine you will need to install Visual Studio on that machine.

In other words, if I would have to decide again 8 months ago, I would probably take NUnit. I may not have the integrated test results report, but developers would have a more seamless testing experience.

Answer 6

Here is my experience with MS Test

• We are running MS Test with around 3800 Test.
• It takes very long for the tests just to start executing, which is painful when running single tests.
• It takes around 1GB Memory to execute the tests. No, it is not due to memory leaks in our tests. Frequently we run into OutOfMemoryExceptions.
• Because it uses that much resource, we are starting to execute the tests from batch-files. So what's the whole integration good for?
• It is buggy and unstable:
  • For instance, if you remove the [Ignore] Attribute from a test, it does not recognize it, because it caches information about tests somewhere. You need to refresh the testlist, which sometimes solves the problem, or restart VS.
  • It randomly does not copy reference assemblies to theout directory.
  • Deployment Items (additional files to be used) just don't work properly. They are ignored randomly.
• There is hidden (not visible in the test code) information in vsmdi and testrunconfig files. If you don't care about it, it might not work.
• Functionally it might be comparable to NUnit, but it is very expensive if you consider using VS tester edition.

---

Addition: We have some more tests now, can't even say how many. It is impossible to run them all anymore from Visual Studio, because of OutOfMemoryExceptions and other instability problems. We run the tests from scripts. It would be easy to view test results in Visual Studio, but when the solution is open, VS crashes (every time). So we need to search the failing tests using text search. There is no advantage of an integrated tool anymore.

Answer 7

You should take a look at Visual T#. It is a language based on C# but with new keywords and logic that enables you to create tests very easily.

You can also look at en.wikipedia.org/wiki/Visual_T_SharpWikipedia for an overview of the capabilities of Visual T#.

Here is some benefits of Visual T#:

• Transparent usage of best practices : clearly identifies the three parts inherent to a test (Preparation, Execution, Verification).
• Fast problems identification : tests that are not well constructed are separated from those that actually failed.
• Easy validation specification : one keyword assert to handle all the cases.
• Effective debugging : tests concerning a working declaration can be executed without knowing where they have been declared.
• Usage of different contexts: reexecute tests for different contexts without rewriting them.
• Indication of missing logical tests: missing logical tests are indicated upon compilation of the test code.
• Simplified writing: each verification can be expressed in one line of code (this also holds for events, exceptions, etc.), use relative verifications.

Answer 8

Biggest advantage MS-Test over NUnit is MS-Test can generate mock objects using Reflection. I found it very usefull