How to diagnose "TestFixtureSetUp Failed"

Question

We use TeamCity as our CI server, and I've just started seeing "TestFixtureSetUp Failed" in the test failure window.

Any idea how I go about debugging this problem? The tests run fine on my workstation (R# test runner in VS2008).

Answer 1

I'd check the Build Log first.

If it's not obvious from that, you could try including Console.WriteLines in the tests - I'm not positive, but I think those are written to the Build Log. Alternately you could log to a file (even using log4net if you wanted to get fancy).

If you have Visual Studio installed on the CI server, you could try running the build/tests from there. If it's a connectivity issue, that might resolve it.

I've seen path issues, though, where relative paths to files were no longer correct or absolute paths were used. These are harder to debug, and might require logging the paths and then checking if they exist on the build server.

Answer 2

It is a bit of a flaw in the implementation of TestFixtureSetUp (and TestFixtureTearDown) that any exceptions are not well reported. I wrote the first implementation of them and I never got it to work the way it was supposed to. At the time the concepts in the NUnit code were tightly coupled to the idea that actions were directly related to a single test. So the reporting of everything was related to a test result. There wasn't really a space for reporting something that happened at the suite level without a huge re-write (it isn't a refactoring when you change a sheep into an escalator).

Because of that bit of history it's hard to find out what really happened in a TestFixtureSetUp. There isn't a good place to attach the error. The TestFixtureSetUp call is a side effect of running a test instead of being directly related to it.

@TrueWill has the right idea. Check the logs and then modify the test to add more logging if necessary. You might want to put at try/catch inside the TestFixtureSetup and log a lot in the catch block. I just thought I could add some background to it (in other words it's kind of my fault).