views:

411

answers:

6

Hi,

QUESTION - If I'm starting a Windows Forms application in C# using Visual Studio 2008, which unit testing framework should I use?

There seems to be one build into VS2008? Or should I look into something like NUnit? Or is NUnit what is used in VS2008 under the bonnet? What's the most popular.

Would be good to have something that allows for: (a) mocking/stubs, and (b) ability to assert exceptions

thanks

+2  A: 

i would say, for simplicity, start with the one built into visual studio 2008. NUnit is fantastic and i use it alot, so you can probably move on to NUnit once your comfortable with writing unit tests.

Andrew Keith
does VS2008 unit tests allow you to assert that an exception should occur? can't quite even see this?
Greg
Yes, although you do it with an attribute ([ExpectedAssertion()] I believe)
Steve
+7  A: 

I would recommend using NUnit for your testing framework. It's very lightweight and easy to ship around if you need to get it set up on a build server. This isn't the case using MSTest. As for mocking/isolation frameworks, Rhino Mocks has the largest user base and you're likely to find answers to your questions quickest with Rhino Mocks.

Chris Missal
The biggest disadvantage with MSTest is once you start down that road it's hard to change your mind. NUnit is a little easier to convert to MSTest. DO try them both.
csharptest.net
NUnit also runs on Mono
joemoe
+9  A: 
Dale Halliwell
Seems unfair when a lot of people use TestDriven.NET to drive NUnit. That provides great integration and removes most of the drawbacks you mention. The NUnit [Ignore] attribute disables [Test] methods or [TestFixture] classes. Also, IMHO ordering test cases is a bit smelly :)
Si
ouch. 600 chars is not going to be enough.1. that's good. I can run my tests on any (build/QA) machine without having to install VSTS. 2. If you want right-click and run tests, get Resharper/TestDriven.Net add-ins. Also see SO q#196740. #3,4,6,10 - doesn't make sense. I believe programmers know their code better than VS. It follows that they can better identify what tests to write. But then I'm test-infected (read TDD zealot). test cases should not be order dependent - so not sure why that is a disadvantage.
Gishu
Contd... Debugging test cases can be done (add nunit as external exe to debug for your test dll) ; although that it implies that you don't know your code anymore. Sorting test cases - again not sure why you need it. But you could group your tests into categories like [Category("Slow Tests")] and then include or exclude them from test runs. You also have a checkbox view where you can manually tick tests you'd like to run. Finally 9. stacktrace - nunit does that in the first tab. A unit testing framework would be kinda crippled if it didnt point to the exact location where the test failed!
Gishu
"Installation of NUnit comes in a separate MSI." You can just reference the assembly from a project. This is how I always use it.
Chris Missal
-1 since so many "facts" are simply wrong (as already pointed out in other comments).
The Chairman
I am also a TDD person, I see this answer has been voted down a number of times. I didn't mean to upset the one-eyed NUnit fanatics, just explain what I used and why it works for me especially for Winforms apps. The comparison comes from the site I linked to. Apart from Gishu who says NUnit can indeed print a stacktrace when a test fails (that does seem kinda critical) the rest are just opinions or justifications for why NUnit does things differently.
Dale Halliwell
+6  A: 

We've started with MSTest. It has an advantage that is (mildly) difficult to replicate in NUnit. It allows you to access private members of the class. This is invaluable when you are checking state. For instance, say that I have a function that puts entries into a dictionary that is not exposed. My understanding, is that with NUnit you have to use reflection to get to the dictionary, or add a getter for the sole purpose of testing. Here, you can just check the dictionary. Very easy and very clean. It also allows you to test private functions, which I love (I know some people don't believe in this). While I don't love MSTest, that feature makes testing much easier.

VS integration is also nice.

Steve
how does MSTest achieve the "access private members" Steve do you know? just interested in why it would be easy for MSTest guys to provide this but not the NUnit guys
Greg
my understanding is the nunit guys don't believe in it. With MSTest, you just right click in the class you want to test and choose "Add Class Accessor". You then access class Foo as Foo_Accessor in your test.
Steve
My group is relatively new to unit testing, and there was a lot of things we liked about NUnit, but the private accessor stuff was a deal breaker.
Steve
It's also an encapsulation breaker.. You're walking right into an anti-pattern - Visit http://stackoverflow.com/questions/333682/tdd-anti-patterns-catalogue and look for inspector. NUnit is doing the right thing by not supporting this.
Gishu
We can agree to disagree on this one. My classes hold stuff and have state. I need to find a way to check that state without calling other functions (if I can help it). I've written the "perfect" classes where every function is self contained and has little/no state, but I haven't been able to do it for classes that actually do stuff.
Steve
Not trying to be dogmatic. If you haven't already read this take a look - http://www.pragprog.com/articles/tell-dont-ask . Peeking inside a class to look at its state - makes tests fragile. Your test now has implementation level knowledge.. tests should state what (and not how). The risk here is that future refactorings e.g. changing the type of variable holding the state or its name - break tests. Hence the term brittle or fragile tests. The behavior is not broken but the tests would still fail.
Gishu
+4  A: 

Use MSTest. It's built into VS2008 and has support in the IDE for creating test functions. If and when you "outgrow" MSTest, then you should look at NUnit or the more modern xUnit. You can expect exceptions in MSTest by decorating with the ExpectedException attribute.

[Test]
[ExpectedException(typeof(ArgumentNullException))]
public void Should_throw_ArgumentNullException_when_the_Order_is_null()
{
    OrderProcessor processor = new OrderProcessor();
    processor.ProcessOrder(null);
}

Mocking frameworks are independent of your choice of testing framework. Popular choices for mocking framework are: Moq and Rhino Mocks.

Phillip Ngan
This can be done with nUnit too.
mberube.Net
For sure. And you can also run NUnit from within Visual Studio if you have resharper.
Phillip Ngan
MSTest doesn't use the [Test] attribute. This is used in NUnit and MbUnit, which makes those two interchangeable, also a bonus for not using MSTest, you're more strongly tied into a framework.
Chris Missal
+1  A: 

You didn't mention if you were going to be test-driving code..

  • Go with NUnit if you're not familiar with any of the xUnit frameworks. Smallest learning curve. It has been around for longer. Charlie Poole is active on NUnit (open source) and has a good history of good updates.
  • VS2008 has MSTest integrated inside it. It's not the same as NUnit. I've never had reason to switch from NUnit. I'd argue that NUnit definitely has the biggest user-base.
  • NUnit has something bundled for Mocks but I'd definitely point you to Moq. Rhino Mocks has been the top dog for a while now but Moq has an easier learning curve (from online docs) although it may not always handle edge scenarios... yet IMHO
  • Asserting exceptions is supported in both.

I'd recommend getting a thin book 'Pragmatic Unit Testing in C# with NUnit' and going through it atleast once.

On the NUnit v MSTest line, I'm not fit to comment. 0 flying-time with MSTest.

  • Although I hear that VS2010 will offer trinkets like automated code-coverage reports.. only if you've MSTest tests. But that's the kind of argument I keep hearing in MSTests favor.. more on the lines of IDE Support and integration. There are great add-ins like Resharper and TestDriven that can patch that in but at a price.
  • Also IMHO, Microsoft's unit testing tooling has a subtle way of leading you down the dark path. so be careful. See my comment to Dale on this thread.
Gishu