Run Visual Studio Unit Tests vs Run ReSharper Unit Tests, differences?

Question

So I have been running into all kinds of interesting problems in VisualStudio 2008 when running Unit Tests.

Such as, when running Visual Studio Unit Tests some tests are failing together but passing individually. This is happening because some class level variables in that test class is being reused in the Unit Tests.

Now normally I would go into each class and fix this problem manually! However, we are talking about tests that range in the thousands!

Now here comes the interesting dilemma, using both ReSharper Unit Tests and TFS BuildServer they are passing together!

Is there any way that I could configure the VS Unit Test Solution to run in the same fashion? I want to avoid calling the [TestInitialize] methods in the [TestCleanup] methods.

Answer 1

This is usually a byproduct of differently ordered tests. ReSharper 4.x and earlier runs unit tests based on the order they appear in the source file. Almost all other unit test runners run tests in alphabetical order. This different ordering can (but never should) affect whether or not tests pass/fail (based on left over data in a database or statics).

ReSharper 5.0 is not using a custom runner anymore so it should fix these inconsistencies.

However, this type of inconsistency indicates a problem in the tests. Some are leaving data behind that they should be cleaning up and some are dependent on, or hurt by, data left over from a previous test.