MS Test error when running all tests

Question

I am currently working on a project that is using MS Test for unit testing. When I do a "Run All Tests" I get the following error for about 1/3 of the tests:

Test method [Test Method] threw exception System.IO.FileLoadException, but exception System.InvalidOperationException was expected. Exception message: System.IO.FileLoadException: Loading this assembly would produce a different grant set from other instances. (Exception from HRESULT: 0x80131401)

If I go to any of the failing tests and Run the test by itself it will give the same error. If I put a break point in the test and debug the test it will pass with no errors. If I again run the individual test it will pass. If I go back to running all tests I get the above error for 1/3 of the tests again.

I had this problem before and I didn't do anything to fix it and it just magically went away. But now it is back and very frustrating.

What is causing this error? Is there a fix for this error?

Answer 1

It sounds like you have Interacting Tests - an xUnit Test Patterns smell.

In short, some tests are dependent on previous tests to have executed, so when you run them in isolation, they change behavior because their implicit assumptions about their environment turn out to be wrong.

This could also explain why you had the problem before, and it then went away. Although MSTest seems to be fairly stable in how it orders tests, it may decide to run them in a different order the next time.

I can't tell you how to resolve the problem as it is individual. However, look for Shared Fixtures. Examples include

• Databases
• Files
• Static (Shared in Visual Basic) types

In your case, the FileLoadException suggests that your tests expect some files to be around. When you run the entire test suite, those files have been left by previous test cases, while they are noticably absent when the test is executed in isolation.