Silverlight Unit Testing

Question

I've searched for this info and see similar questions, but not one that matches very closely to this. If I missed one, I apologize. I was hoping that you'd be able to point me in a direction. I'm working on a Silverlight based project and my team is finally moving toward implementing unit testing. I and another memeber of my team are responsible for recommending a unit testing framework for the project. Our basic criteria are:

1. The project contains standard ASMX .NET webservices and a Silverlight front end. We'd prefer, if at all possible, to maintain the same test attributes throughout, rather than use one set for Silverlight tests and another for other code.

2. Integration with VS 2008 is fairly important. We'd like to keep it all under one roof if that's feasible. We'd be happy with simply being able to kick the unit test off from VS.

3. Automated build/checkin testing. We are currently working in a completely non-automated VS 2008/VSS 2005 environment. We are in the process of converting to SVN for source control and our corporate office will be assisting us in using MSBuild to automate the build process. We'd certainly prefer to be able to integrate into this environment as much as possible. I'm not certain as to the details of this process as yet since I'm not directly involved. If there is more detail that you need on this, please let me know and I'll see what I can find out.

At this point, my collegue and I are looking at NUnit (along with possble Silverlight options) and MSTest in conjunction with the Silverlight Testing Framework that Jeff Wilcox wrote. While the corpoarate standard is NUnit, they are open to other options as there aren't any other teams doing Silverlight work.

I'm about halfway through Roy Osherove's Unit Testing Book, so I'm getting a feel for writing tests in general. I'm not married to a particular framework, but corporate seems to be most open to NUnit or MSTest. I'd also like to get my hands on a good tutorial of MSTest, if possible.

Thanks for taking the time. If you need any other info from me, feel free to ask.

Cheers,

Steve

Answer 1

I'm not going to recommend a unit testing framework, but I can warn that the Test attributes in MSTest are sealed (unable to be inherited in a new custom attribute which you create), unlike NUnit. In my mind, that fact shouldn't matter too much because I also suggest that, whichever testing framework you end up with, stick with its single Test attribute rather than creating your inherited one. To denote special tests (such as Silverlight like you mentioned) you should add an additional attribute. The tricky bit comes in when you want to only run the tests that have your custom attribute applied (as opposed to all of the unit tests), and that would require a custom driver for the tests.

I hope this helps.

Answer 2

Tread carefully, and make sure to experience these possibilities first. It's easiest to know what you're looking at by writing a few tests in each, and figure out what your workflow will be.

The Silverlight unit test systems out there are less mature than the officially shipped, full-fledged test frameworks for the desktop .NET environment, so your experience may vary. Understand that the Silverlight test solutions in place today (that actually run inside of a web browser) exist to specifically help folks get coverage of their code and components within the context of the Silverlight platform, and not necessarily for rapid, or easy, test-driven development and testing that you can get through the integrated Visual Studio tools.

The sandboxed security model for Silverlight makes many of the standard testing tools you may expect much more difficult.

It's also a different platform, really, so it may not make sense to perform every kind of test (example: a "load test" for a web app... you can "stress" a Silverlight app, or look into its "performance", but a "load test" is about stressing your machine that hosts the .XAP file, and is not very interesting for Silverlight apps specifically).

If you're more concerned about testing your business logic and having nice integration, strongly consider writing mockable/IoC code enough that you may be able to develop most of your app and test a version of it built as a regular desktop .NET class library ~ you could then have a subset of tests that are Silverlight in-browser only.

To your notes:

• Are you actually going to be testing the web service calls with your tests? This sort of integration/client + server testing can be very, very difficult to do properly between Silverlight client code and a web service.

• You won't get much integration with Visual Studio 2008's IDE for the Silverlight unit test framework. You can have a Silverlight test project / app that you can press F5 to open, and debug, in Visual Studio and run the tests. However, you won't have right-click "Run All Tests" or test case management tools working.

• Automated test check-in for the Silverlight unit testing framework is something that is developing slowly, some folks have written a set of Silverlight test runners on CodePlex. You may not get this easily, and your build/test machine will need to be setup so that the build service runs instead as an actual user process, so that it can open and control web browser(s)

In the future, on the Silverlight Toolkit team, we are going to release some level of automated test support in the future, along with simple code coverage support, but these tools are not readily available as of today (Oct. '09)