views:

1435

answers:

9

Hi all,

I'm using Visual studio (sometimes resharper) to run my unit test.

I heard about NUnit, but I don't know many things about it...

Should I care about it ? Can it offer something better than visual studio ? Should I Use NUnit and why ?

Thanks for your help

Tim

+1  A: 

NUnit is a unit testing framework, which is also supported by resharper. I think you are using Microsoft's unit testing framework, so NUnit is just an alternative to Microsoft's product ;)

Here is the link to the homepage of NUnit: http://nunit.org/index.php

Oliver Hanappi
so you mean that nunit doesn't bring anything else than microsoft unit testing framework ?
Tim
It does, see my link for instance in my post (above).
bastijn
I'm using NUnit and don't really know about Microsoft's unit testing framework, so I can't say what's better. I think there is a good chance that you will find some topic on the differences here on SO.
Oliver Hanappi
It provides some useful constructs such as [TestCase] for running one test method with different args, [Theory] - for building more elaborate specifications and others.It also supports very nice fluent syntax for assertions. And, last but not the least, it is used much wider than MSTest, so you'd have better chance to get support/info if you'd get into trouble.
elder_george
+6  A: 
bastijn
+17  A: 

NUnit has few advantages over MS-Test

  1. Suite attribute - can aggregate tests and execute them separately (useful for large projects with fast and slow tests for example)
  2. Readable Assert method, e.g. Assert.AreEqual(expected, actual) vs Assert.That(actual, Is.EqualTo(expected))
  3. NUnit has frequent version updates - MS-Test has only one per VS version.
  4. Many integrated runners including Resharper and TestDriven.NET
  5. Expected exception message assertion - can be done using attribute in NUnit but must be done using Try-Catch in MS-Test
Elisha
Exception can also be asserted by attribute in MS-Test: ExpectedExceptionAttribute.
Stefan Steinegger
I would use NUnit with Assert.Throws<>() because this follows the AAA-Principle, which is not through for the attribute way.
Oliver Hanappi
@Stefan Steinegger, MSTest has expected exception attribute but it cannot verify the message text
Elisha
NUnit has a standalone runner thats easy and light to install
Ruben Bartelink
#3 is not a feature, it's a problem and #5 is 100% false; MS Test has the ExpectedException attribute, *and always has*.
Randolpho
@Randolpho, ExpectedException exists, but it doesn't support message assertion. The missing feature is only the thrown exception message assertion, not the thrown exception itself. I can assert using MS-Test attribute if the exception was thrown but if I want to verify it contains the right message it must be done using try-catch.
Elisha
A: 

I am not sure of others but NUnit provides nice GUI and console to run your Unit Tests and also you can generate report of the result of NUnit Test execution which would give the detail of whethere the test has failed or passed and also what time did it take for your unit test

Vinay Pandey
A: 

NUnit works with the Standard edition of VS.

Per Erik Stendahl
Don't forget Visual Studios express versions.
chobo2
Post-build step on the unit test assembly project -- \path\to\nUnit $(TargetFileName)
Steve Gilham
+14  A: 

From my current perspective (after 8 months of development with about 10 developers on average) I would advice against using MSTest for the following reasons

  • The framework in itself is quite slow. I don't mean the test, they are as fast as you write them, but the framework in itself will almost always take more time to run your test suite, single tests etc.
  • The need to keep a Test-Metadata file which always leads to complications when several developers are working on it (recreating e.g. the metadata etc.). Every other test suite doesn't need a metadata file. It is kind of nice to organize your tests but you can achieve the same through namespaces, classes and method names.
  • Doing Continuous Integration, if you want to run unit tests on your build machine you will need to install Visual Studio on that machine.

In other words, if I would have to decide again 8 months ago, I would probably take NUnit. I may not have the integrated test results report, but developers would have a more seamless testing experience.

flq
+1, same experience here.
Stefan Steinegger
+1, avoid MSTest unless you don't have a choice. The various open-source frameworks are better (xUnit, NUnit, MbUnit, etc).
Brannon
+8  A: 

Here is my experience with MS Test

  • We are running MS Test with around 3800 Test.
  • It takes very long for the tests just to start executing, which is painful when running single tests.
  • It takes around 1GB Memory to execute the tests. No, it is not due to memory leaks in our tests. Frequently we run into OutOfMemoryExceptions.
  • Because it uses that much resource, we are starting to execute the tests from batch-files. So what's the whole integration good for?
  • It is buggy and unstable:
    • For instance, if you remove the [Ignore] Attribute from a test, it does not recognize it, because it caches information about tests somewhere. You need to refresh the testlist, which sometimes solves the problem, or restart VS.
    • It randomly does not copy reference assemblies to theout directory.
    • Deployment Items (additional files to be used) just don't work properly. They are ignored randomly.
  • There is hidden (not visible in the test code) information in vsmdi and testrunconfig files. If you don't care about it, it might not work.
  • Functionally it might be comparable to NUnit, but it is very expensive if you consider using VS tester edition.

Addition: We have some more tests now, can't even say how many. It is impossible to run them all anymore from Visual Studio, because of OutOfMemoryExceptions and other instability problems. We run the tests from scripts. It would be easy to view test results in Visual Studio, but when the solution is open, VS crashes (every time). So we need to search the failing tests using text search. There is no advantage of an integrated tool anymore.

Stefan Steinegger
+1  A: 

You should take a look at Visual T#. It is a language based on C# but with new keywords and logic that enables you to create tests very easily.

You can also look at en.wikipedia.org/wiki/Visual_T_SharpWikipedia for an overview of the capabilities of Visual T#.

Here is some benefits of Visual T#:

  • Transparent usage of best practices : clearly identifies the three parts inherent to a test (Preparation, Execution, Verification).
  • Fast problems identification : tests that are not well constructed are separated from those that actually failed.
  • Easy validation specification : one keyword assert to handle all the cases.
  • Effective debugging : tests concerning a working declaration can be executed without knowing where they have been declared.
  • Usage of different contexts: reexecute tests for different contexts without rewriting them.
  • Indication of missing logical tests: missing logical tests are indicated upon compilation of the test code.
  • Simplified writing: each verification can be expressed in one line of code (this also holds for events, exceptions, etc.), use relative verifications.
Ludovic Dubois
+1  A: 

Biggest advantage MS-Test over NUnit is MS-Test can generate mock objects using Reflection. I found it very usefull

Maciej
http://code.google.com/p/moq/ , http://www.nmock.org/ , http://stackoverflow.com/questions/37359/what-c-mocking-framework-to-use
David Schmitt