Assemblies mysteriously loaded into new AppDomains

Question

I'm testing some code that does work whenever assemblies are loaded into an appdomain. For unit testing (in VS2k8's built-in test host) I spin up a new, uniquely-named appdomain prior to each test with the idea that it should be "clean":

[TestInitialize()]  
public void CalledBeforeEachTestMethod()  
{  
 AppDomainSetup appSetup = new AppDomainSetup();  
 appSetup.ApplicationBase = @"G:\<ProjectDir>\bin\Debug";
 Evidence baseEvidence = AppDomain.CurrentDomain.Evidence;  
 Evidence evidence = new Evidence( baseEvidence );  
 _testAppDomain = AppDomain.CreateDomain( "myAppDomain" + _appDomainCounter++, evidence, appSetup );  
}  

[TestMethod]
public void MissingFactoryCausesAppDomainUnload()
{
 SupportingClass supportClassObj = (SupportingClass)_testAppDomain.CreateInstanceAndUnwrap(
GetType().Assembly.GetName().Name,
typeof( SupportingClass ).FullName );  
 try  
 {  
  supportClassObj.LoadMissingRegistrationAssembly();  
  Assert.Fail( "Should have nuked the app domain" );  
 }  
 catch( AppDomainUnloadedException ) { }  
}

[TestMethod]
public void InvalidFactoryMethodCausesAppDomainUnload()
{
 SupportingClass supportClassObj = (SupportingClass)_testAppDomain.CreateInstanceAndUnwrap(
GetType().Assembly.GetName().Name,
typeof( SupportingClass ).FullName );  
 try  
 {  
  supportClassObj.LoadInvalidFactoriesAssembly();  
  Assert.Fail( "Should have nuked the app domain" );  
 }  
 catch( AppDomainUnloadedException ) { }  
}

public class SupportingClass : MarshalByRefObject
{
 public void LoadMissingRegistrationAssembly()  
 {  
  MissingRegistration.Main();  
 }  
 public void LoadInvalidFactoriesAssembly()  
 {  
  InvalidFactories.Main();  
 }  
}

If every test is run individually I find that it works correctly; the appdomain is created and has only the few intended assemblies loaded. However, if multiple tests are run in succession then each _testAppDomain already has assemblies loaded from all previous tests. Oddly enough, the two tests get appdomains with different names. The test assemblies that define MissingRegistration and InvalidFactories (two different assemblies) are never loaded into the unit test's default appdomain. Can anyone explain this behavior?

Answer 1

It sounds like what's is happening is that the assemblies are getting loaded in the parent AppDomain. If so, your bug is in the details of how you use are using _testAppDomain elsewhere in your test code, now how you create it.

Ideally test harness code should run itself in the AppDomain, and then the method running in the AppDomain should do the actual loading of the assemblies under test.

Here is an example to give the idea of how to keep the parent AppDomain from ever loading your test assemblies:

void TestRunner()
{
  testProxy =
    (TestProxy)_testAppDomain.CreateInstanceAndUnwrap(
                      typeof(TestProxy).Assembly.FullName,
                      typeof(TestProxy).FullName)

  testProxy.RunTest(testAssembly, typeName);
}

public class TestProxy : MarshalByRefObject
{
  public void Runtest(string testAssembly, string typeName)
  {
    var testType = Assembly.Load(testAssembly).GetType(typeName);

    // run tests in testType using reflection or whatever
  }
}

In your particular situation, however, it may be that you don't really have any test assemblies per se, so maybe this doesn't apply.

I also noticed that your [TestInitialize] comment says it is called once per test, but IIRC, the docs say Visual Studio's test framework only calls this once per class when it is running multiple tests. I use a different framework, so I am not sure of this.

Update

Now that I can see the rest of your code, I can see you already are taking reasonable precautions not to load assemblies in the parent AppDomain. You say that is indeed not happening, and I believe you. If worse comes to worse, you could try having your SupportingClass call another assembly which then does your test, but I don't really think this would change anything.

I do have another theory for you:

I read somewhere (on a blog, I think) that JITed methods are cached and reused between AppDomains based on a signature that includes some of the assembly load rules. I assume this would include ApplicationBase.

So my theory is that when NET Framework loads your assembly (or maybe when it loads your SupportingClass), it is scanning for already-JITed methods it can use. When it finds the previously JITed method it "enables" it (for lack of a better word) in that AppDomain, which triggers a load of assembly it depends on.

This would explain why changin ApplicationBase makes it work: The JITed methods would not be reused from the cache, and since the method is never called it is not JITed again so the dependent assembly is never loaded.

It seems to me that you have a good workaround in varying the ApplicationBase, or you might try varying the Evidence if the ApplicationBase is important to keep the same. I would think this would have the same effect.