NUnit vs Visual Studio 2008's Test Projects for Unit Testing?

Question

I am going to be starting up a new project at work and want to get into unit testing. We will be using VS 2008, C#, and the ASP.NET MVC stuff. I am looking at using either NUnit or the built in test projects that VS2008 has, but I am open to researching other suggestions. Is one system better than the other or perhaps easier to use/understand than the other? I am looking to get this project set up as kind of the "best practice" for our development efforts going forward.

Thanks for any help and suggestions!!

Answer 1

I have been using NUnit for 2 years. All is fine but I have to say that the Unit system in VS is pretty nice because it's inside the Gui and can more easily do test for private function without having to mess around. Also, the Unit Testing of VS let you do covering and other stuff that NUnit alone can't do.

Answer 2

.NET Testing Framework Advice and .NET Unit Testing packages?.

Answer 3

Daok named all the pro's of VS2008 test projects, here are the pro's of NUnit.

• NUnit has a mocking framework.
• NUnit can be run outside of the IDE, this can be useful if you want to run tests on a non MS build server like CC.Net
• NUnit has more versions coming out than visual studio. You don't have to wait years for a new version And you don't have to install a new version of the IDE to get new features.
• There are extensions being developed for NUnit like row-tests etc.
• Visual Studio tests take a long time to start up for some reason. This is better in 2008 but still too slow for my taste. Quickly running a test to see if you didn't break something can take too long. NUnit with something like Testdriven.Net to run tests from the IDE is actually much faster. especially when running single tests.
  Accorting to Kjetil Klaussen this is caused by the Visual Studio testrunner, running MSTest tests in TestDriven.Net makes MSTest performance comparable to NUnit.

Answer 4

The unit-testing framework doesn't actually matter much, because you can convert test classes with separate project files and conditional compilation (like this, VS->NUnit):

 #if !NUNIT
  using Microsoft.VisualStudio.TestTools.UnitTesting;
 #else
  using NUnit.Framework;
  using TestClass = NUnit.Framework.TestFixtureAttribute;
  using TestMethod = NUnit.Framework.TestAttribute;
  using TestInitialize = NUnit.Framework.SetUpAttribute;
  using TestCleanup = NUnit.Framework.TearDownAttribute;
  using TestContext = System.String;
  using DeploymentItem = NUnit.Framework.DescriptionAttribute;
 #endif

The TestDriven.Net plugin is nice and not very expensive... With only plain VS2008 you have to find the test from your test class or test list. With TestDriven.Net you can run your test directly from the class that you are testing. After all, unit test should be easy to maintain and near the developer.

Answer 5

XUnit is another possibility for a greenfield project. It's got perhaps a more intuitive syntax, but is not really compatible with the other frameworks.

http://www.codeplex.com/xunit

Answer 6

First I want to currect a wrong statement: you can run msTest outside of visual studio using command line. Although several CI tools such as TeamCity have better support for NUnit (probably would change as msTest become more popular). In my current project we use both and the only big diffrence we found is that mstest always runs as a 32bit while NUnit runs as either 32bit or 64bit test which only metter if your code uses native code that is 32/64 dependant.

Answer 7

As far as I know, there a four frameworks available for unit testing with .NET these days

• NUnit
• MbUnit
• MSTest
• xUnit

NUnit has always been out in front but the gap has closed in the last year or so. I still prefer NUnit myself, especially as they added a fluent interface a while back which makes tests very readable.

If you're just getting started with unit testing it it probably doesn't make much difference. Once you're up to speed you'll be in a better position to judge which framework is best for your needs.

Answer 8

One slight annoyance of Visual Studio's testing framework is that it will create many test run files that tend to clutter your project directory - though this isn't that big of a deal.

Also, if you lack a plugin such as TestDriven.NET, you cannot debug your NUnit (or MbUnit, xUnit, etc.) unit tests within the Visual Studio environment, as you can with the Microsoft VS testing framework, which is built in.

Answer 9

Benefits/changes of VS2008 Built-in Unit Testing Framework

1. The 2008 version now is available in professional editions (before it required expensive versions of VS, this is just for developer unit testing.) that left a lot of developers with the only choice of open/external testing frameworks.
2. Built in API supported by single company.
3. Use the same tools to to run and create tests (you may run them using the command line also MSTest)
4. Simple design (granted no Mock framework, but this is a great starting point for many programmers)
5. Long term support granted (I still remember what happened to nDoc, I don't want to commit to a testing framework that might not be supported in 5 years, but I still consider nUnit a great framework.)
6. If using team foundation server as your backend, you can create work items or bugs with the failed test data in a simple fashion.

Answer 10

Slightly off-topic, but if you go with NUnit I can recommend using ReSharper - it adds some buttons to the VS UI that make it a lot easier to run and debug tests from within the IDE.

This review is slightly out-of-date, but explains this in more detail:

http://codebetter.com/blogs/paul.laudeman/archive/2006/08/15/Using-ReSharper-as-an-essential-part-of-your-TDD-toolkit.aspx

Answer 11

MSTest is essentially NUnit slightly reworked, with a few new features (such as assembly setup and teardown, not just fixture and test level), and missing some of the best bits (such as the new 2.4 constraint syntax). NUnit is more mature, and there is more support for it from other vendors; and of course since it's always been free (whereas MSTest only made it into the Professional version of 2008, before that it was in way more expensive SKUs), most ALT.NET projects use it.

Having said that, there are some companies who are incredibly reluctant to use something which does not have the Microsoft label on it, and especially so OSS code. So having an official MS test framework may be the motivation that those companies need to get testing; and let's be honest, it's the testing that matters, not what tool you use (and using Tuomas Hietanen's code above, you can almost make your test framework interchangeable).

Answer 12

I got messages that "NUnit file structure is richer than VSTest"... Of course if you prefer the NUnit file structure, you can use this solution to the other way, like this (NUnit->VS):

 #if !MSTEST
  using NUnit.Framework;
 #else
  using Microsoft.VisualStudio.TestTools.UnitTesting;
  using TestFixture = Microsoft.VisualStudio.TestTools.UnitTesting.TestClassAttribute;
  using Test = Microsoft.VisualStudio.TestTools.UnitTesting.TestMethodAttribute;
  using SetUp = Microsoft.VisualStudio.TestTools.UnitTesting.TestInitializeAttribute;
  using TearDown = Microsoft.VisualStudio.TestTools.UnitTesting.TestCleanupAttribute;
 #endif

Or any other conversion... :-) This using here is just alias to compiler.

Answer 13

My main beef with VS unit tests over NUnit is the VS test creation tends to inject a bunch of generated code for private member access.

Some might want to test their private methods, some may not, that's a different topic.

My concern is when I'm writing unit tests they should be extremely controlled so I know exactly what I'm testing and exactly how I'm testing it. If there's auto generated code I'm losing some of that ownership.

Answer 14

I would prefer to use MS's little test framework, but for now am sticking with NUnit. The problems with MS's are generally (for me)

• Shared "tests" file (pointless) that must be maintained
• Tests lists cause conflicts with multiple developers / VCSs
• Poor integrated UI - confusing setup, burdensome test selection
• No good external runner

Caveats - If I were testing an aspx site, I would definitely use MS's - If I were developing solo, also MS would be fine - If I had limited skill and couldn't configure NUnit :)

I find it much easier to just write my tests and fire up NUnitGUI or one of the other front ends (testDriven is far far far far overpriced). Setting up debugging with the commandline version is also pretty easy.

Answer 15

I started with MSTest but switched for one simple reason. MSTest does not support Inheritance of Test Methods from other assemblies.

I hated the idea of writing the same test multiple times. Especially on a large project where test methods can easily run into 100's of tests.

NUnit does extactly what i need. The only thing that is missing with NUnit is a Visual Studio Addin which has can display the Red/Green status (Like VSTS) of each test.

Answer 16

I have done some TDD using both and (maybe I'm a little dumb) nUnit seems to be a lot faster and simpler to use to me. And when I say a lot, I mean a lot.

In MS Test, there is too many attributes, everywhere - the code that do the real tests is the tiny lines you may read here and there. A big mess. In nUnit, the code that do the test just dominates the attributes, as it should do.

Also, in nUnit, you just have to click on the tests you want to run (only one? all the tests covering a class? an assembly? the solution?). One click. And the window is clear and large. You get clear green and red lights. You really know what happens in one sight.

In VSTS, the test list is jammed in the bottom of the screen, it's small and ugly. You have to look twice to know what happened. And you cannot run just one test (well, I did not find out yet!).

But I may be wrong, of course - I just read about 21 blog posts about "How to do simple TDD using VSTS". I should have read more, you are right.

For nUnit, I read one. And I was TDDing the same day. With fun.

By the way, I usually love Microsoft products. Visual Studio is really the best tool a developer can buy - but TDD and Work Item management in Visual Studio Team System sucks, really.

All the best. Sylvain.

Answer 17

If you are considering either MSTest or nUnit, then I recommend you look at mbUnit. My reasons are

1. TestDriven.Net compatibility. Nothing beats have TestDriven.Net.ReRunWithDebugger bound to a keyboard combination.
2. The Gallio framework. Gallio is a test runner like nUnits. The only difference is it doesn't care if you wrote your tests in nUnit, msTest, xUnit or mbUnit. They all get run.
3. Compatibility with nUnit. All features in nUnit are supported by mbUnit. I think you don't even need to change your attributes (will have to check that), just your reference and usings.
4. Collection asserts. mbUnit has more Assert cases, including the CollectionAssert class. Basically you no longer need to write your own tests to see if 2 collections are the same.
5. Combinatorial tests. Wouldn't it be cool if you could supply two sets of data and get a test for all the combinations of data. It is in mbUnit.

I originally picked up mbUnit becasue of its [RowTest ....] functionality, and I haven't found a single reason to go back. I moved all my active test suites over from nUnit, and never looked back. Since then I've converted two different development teams over to the benefits.

Answer 18

I don't like VS built-in testing framework because it forces you to create a separate project as opposed to having your tests as part of the project you're testing.