How can I run NUnit(Selenium Grid) tests in parallel?

Question

My current project uses NUnit for unit tests and to drive UATs written with Selenium. Developers normally run tests using ReSharper's test runner in VS.Net 2003 and our build box kicks them off via NAnt.

We would like to run the UAT tests in parallel so that we can take advantage of Selenium Grid/RCs so that they will be able to run much faster.

Does anyone have any thoughts on how this might be achieved? and/or best practices for testing Selenium tests against multiple browsers environments without writing duplicate tests automatically?

Thank you.

Answer 1

There hasn't been a lot of work on this subject. I didn't find anything really relevent.

However, your point is well taken. Most machines nowadays have more cores and less powerful cores compared to powerful one core cpu.

So I did find something on a Microsoft blog. The technology is called PUnit and is made especially for testing multi-threaded environment.

It's as close as possible to what you requested that I could find :)

You can visit it the appropriate blog post right there: http://blogs.microsoft.co.il/blogs/eyal/archive/2008/07/09/punit-parallel-unit-testing-in-making.aspx

Update: Link is not valid anymore. The project on CodePlex has been removed.

Update2: This is on the roadmap for NUnit 2.5. Reference

Answer 2

Thank you Maxim for your suggestion.

I did look at PNUnit briefly but had forgotten about it. My main reservation for it was the need to enumerate each individual tests (not just test fixture) in the configuration XML files. We could automate the generation of these files but I was hoping for a bit more of a streamlined solution.

After a discussion with the team, we really have two issues.

1) The build server should run all UATs against all browsers, reporting the results seperately (or at least noting which errors came from which browser), and run them in parallel (assuming there are RCs available) so that the build doesn't take forever.

2) Developers would like an easier way of kicking off a test or group of tests against some subset of the browsers without rebuilding the project and without a lot of other manual changing.

Also, if multiple RCs are available, ideally same browser tests could run multiple at a time to speed things up

My best idea for problem 1 is an individual cruise build for each browser environment that run independently. This seems like overkill but is the simplest solution I could come up with and should work.

For problem 2 I don't yet have a solution. I have considered writing a custom Resharper plugin to allow for concurrent execution and/or the setup of environment variables on a session of tests so that each session can be configured to run against a particular environment without effecting another. However I don't know yet if this is worth the time to learn/develop the plugin.

Answer 3

I struggled with both these problems myself. In the end I developed a custom Nunit test runner that is capable of running multiple tests in parrallel. This combined with the Taumuon.Rakija extension for nunit allowed the tests to be dynmaically created depending on which browser you want the test to run on.

I'm now in a position where I can launch my test suite against as many browser types on as many operating systems as I wish in parrallel.

Unfortunately there doesn't seem to be a good solution to these problems already so you'll probably have to solve them yourself for your particular environment.

Answer 4

What about NUnit 2.5? www.nunit.org/index.php?p=releaseNotes&r=2.5

It seems to support parallel run.

Answer 5

Igor Brejc has a blog post about running tests in parallel using MbUnit.

However he does say "Once we integrate Selenium into acceptance testing", so it looks like he was just experimenting, I can't find any other posts so I don't know if he has successfully run Selenium tests in parallel.

http://igorbrejc.net/development/continuous-integration/gallio-running-tests-in-parallel

Answer 6

here is the sample of ASP.NET application tested with selenium and mbunit. All tests can be runned in parallel.

http://code.google.com/p/design-of-selenium-tests-for-asp-net/

description is here

http://slmoloch.blogspot.com/2009/12/design-of-selenium-tests-for-aspnet_19.html

Sorry, i still cant contribute links.