Is there anything I can do in NUnit that I can't do in MSTest?

Question

This question has been asked in various forms in a number of different forums, but, IMHO, I haven't been able to find a place where it's really answered clearly, so I'm going to reframe it and ask it again.

I work in a basically Microsoft Shop. We use TFS, and all of our developers have MSDN subscriptions including the Team Suite edition of VS. So we have access to MSTest.

I've read the various NUnit vs. MSTest comparisons, and the developer community seems to pretty much overwhelmingly choose NUnit. But the reasons given don't ever seem to be overwhelming or compelling, at least to our situation. (NUnit is updated more often, NUnit is faster, NUnit doesn't require TFS, etc.)

I can use NUnit if I choose, but the use of open source software without a formal support behind it has to be defended. I need a fairly compelling reason to do so.

What I basically have to answer to justify using NUnit in preference to MSTest is this: is there anything that I can do in NUnit that I can't do with comparable effort in MSTest?

Answer 1

NUnit has a richer assert API. The api is particularly elegant (fluent, even), for example

Assert.That(Is.Unique, myResults);  // assert: myResults is a collection of unique items

If you've seen the Hamcrest extensions to JUnit you'll recognise this style.

It also has a growing set of extensions, such as performance testing and an excellent VS plugin.

Answer 2

I can point you to a couple of blogs on frustrations with MSTest:

• MSBuild, NAnt, NUnit, MSTest, and frustration
• Clean build server with MSTest == FAIL

To be fair, these people are trying to set up MSTest on non-TFS build servers. Some of their issues are not going to apply to your situation.

We are primarily a Microsoft shop, and use TFS for source control. However, we use TeamCity for Continuous Integration; we like it, and it integrates reasonably well with TFS. I've never used MSTest; we've been using NUnit for years, and have seen no reason to change.

MSTest is supposed to have tight integration with Team Suite, which (since your company has already paid the outrageous fee for that) is a point in its favor.

NUnit comes with less vendor lock-in, and has a rich API. As serg10 pointed out, the Assert.That syntax is particularly powerful and elegant.

In the end, you can write good unit tests without all the fancy features. Some of them may even get in the way (which is the theory behind xUnit.net). I would recommend that your team standardize on one test framework; avoid having some code in MSTest and other code in NUnit.

I think writing good tests is more important than your choice of frameworks. Consider reading The Art of Unit Testing: with Examples in .NET, writing some tests, then seeing if MSTest is adequate for your team's needs.

EDIT: Appendix B of The Art of Unit Testing has some good comments on Microsoft's Unit Testing Framework. It mentions YUnit as an example of how cumbersome it is to extend MSTest. However, the author does suggest MSTest for Team System users due to the tight integration.

Answer 3

(update - see Euan's answer below later in the page for some corrections to this answer)

• NUnit contains a [TestCase] attribute that allows implementing parametrized tests. this does not exist in MSTest
• MsTest's ExpectedException attribute has a bug where the expected message is never really asserted even if it's wrong - the test will pass.
• NUnit has an Assert.Throws API to allow testing an exception on a specific line of code instead of the whole method (you can easily implement this one yourself though)
• NUnit contains a fluent version of Assert API (as already mentioned - Assert.That..)
• NUnit is much faster
• NUnit can run tests in 32 and 64 bit (MSTest only runs them in 32 bit IIRC)
• NUnit allows abstract classes to be test fixtures (so you can inherit test fixtures). MsTest does not.
• NUnit allows non public classes to be test fixtures (as of the latest version)
• NUnit was created SOLELY for the idea of unit testing. MSTest was created for Testing - and also a bit of unit testing.
• NUnit contains PNunit (running parallel tests with NUnit). MSTest only adds this ability in vs 2010

Answer 4

I've been working on first class support for MSTest in TestDriven.Net 3.0. In previous versions the MSTest support was very basic (there seemed to be few people using MSTest).

As of TestDriven.Net 3.0 Beta 2, there is pretty comprehensive support for all of the unit testing related attributes of MSTest. There is even support for data driven tests using the DataSource attribute. Your tests will also execute at close to NUnit speeds!

If you use MSTest, I'd be interested to hear if any of your unit tests fail when executed using TestDriven.Net 3.0 Beta 2 (or later).

Please try it on your MSTest projects and let me know how you get on: http://www.testdriven.net/download.aspx

NOTE, it doesn't support the DeploymentItem or HostType attributes. You can use the 'Copy to Output Directory' project item setting instead of DeploymentItem.

Answer 5

You can change the code in NUnit because it is open source. You can't do that with MSTest.

Answer 6

Roy, Bunch of your information is out of date especially as it relates to 2010;

Nunit contains a [TestCase] attribute that allows implementing parametrized tests. this does not exist in MSTest

This can be implemented using unit test extensibility in 2010.

MSTest's ExpectedException attribute has a bug where the expected message is never really asserted even if it's wrong - the test will pass.

Correct thats still there

NUnit has an Assert.Throws API to allow testing an exception on a specific line of code instead of the whole method (you can easily implement this one yourself though)

Jim implented a version of Assert.Throws for MSTest at the same time as he did the original implementation for NUnit, NUnit has included in subsequent releases, MSTest has not, its still possible to use though.

NUnit contains a fluent version of Assert API (as already mentioned - Assert.That..)

There are several of these implemented by 3rd parties for MSTest

NUnit is much faster

See Jamie's comment he has managed to get MSTest running faster :-)

NUnit can run tests in 32 and 64 bit (MSTest only runs them in 32 bit IIRC)

Not in 2010, 64 bit support is built in.

NUnit allows abstract classes to be test fixtures (so you can inherit test fixtures). MsTest does not.

This works but not across assemblies which does limit its usefulness.

NUnit allows non public classes to be test fixtures (as of the latest version)

Still there

NUnit was created SOLELY for the idea of unit testing. MSTest was created for Testing - and also a bit of unit testing.

Correct, there is a lot of misconception that MSTest is the same as Nunit but MSTest is a generalized framework.

NUnit contains PNunit (running parallel tests with NUnit). MSTest only adds this ability in vs 2010

Correct there is an XML config setting that allows control of degree of parallelism.

Answer 7

I have a nice way in between MsTest and NUnit. You can use the MSTest framework to run your test (TestClass attribute and TestMethod attribute for each test) but use the NUnit Assert API.

just do this :

using Microsoft.VisualStudio.TestTools.UnitTesting; 
using Assert = NUnit.Framework.Assert;  

now you can use Assert.That(..) and still have TFS automated build and test report. your test can be run in VS, TestDriven.net or Resharper, it doesnt matter, the test will fail or pass correctly, and the fail output will be according to NUnit framework.

see here : http://alsagile.com/archive/2010/03/09/stop-the-war-between-nunit-and-mstest-make-them.aspx

Answer 8

See http://fluentassertions.codeplex.com. You can do stuff like

"ABCDEFGHI".Should().StartWith("AB").And.EndWith("HI").And.Contain("EF").And.HaveLength(9);

new[] { 1, 2, 3 }.Should().HaveCount(4, "because we thought we put three items in the 
collection"))

dtoCollection.Should().Contain(dto => dto.Id != null);

collection.Should().HaveCount(c => c >= 3);

dto.ShouldHave().AllPropertiesBut(d => d.Id).EqualTo(customer);

dt1.Should().BeWithin(TimeSpan.FromHours(50)).Before(dt2); 

Action action = () => recipe.AddIngredient("Milk", 100, Unit.Spoon);
action
   .ShouldThrow<RuleViolationException>()
   .WithMessage("Cannot change the unit of an existing ingredient")
   .And.Violations.Should().Contain(BusinessRule.CannotChangeIngredientQuanity