What is currently the best solution to automate running unit tests against silverlight applications within CruiseControl.NET?
Preferably I would like to run MSTest unit tests however as these aren't officially supported for silverlight yet I am open to suggestions that use other testing frameworks.
The November version of the Silverlight Toolkit contains not only the Silverlight Unit Testing Framework, but also an MSBuild task to allow the framework to be run from the command line. See Jeff Wilcox's post for detail, but you essentially end up doing this:
msbuild /t:test /p:browser=firefox
As CC.Net can run MSBuild you should be able to at least call the tests. The output is in TRX format.
I've used this framework with some success, although not integrated into CC.Net, it works fine but the tests run in a browser so are a little slow. The advantage of running in the browser is you can do more acceptance-like tests, running up controls to display. The framework also includes methods to cope with the asynchronous nature of Silverlight.
I've been having this issue and at the moment there's no official supported way of doing these unit tests. The main problem is that the Silverlight Unit Testing Framework does not support plugging into MSTest and therefore can't really be integrated into any CI. However there is a way to hack around this using Powerhell - see here.
I personally see that it's quite a hassle to get these tests plugged into CI and for the time being run them only on my local machine.
Update: as jbloomer wrote, there is a walkthrough available on Jeff Wilcox's blog on how to set up an MSBuild task for the Silverlight Unit Testing Framework. I would recommend choosing this path: once MSBuild is set up, you should have no problem plugging it into CruiseControl.NET.
Could you use a third party testing framework like NUnit or XUnit.net? You could then call the executable from within CC.net. In fact, CCNet has an NUnit task.