Is it possible that COM interop is affected by the Visual Studio Hosting Process?

Question

I am having problems running a 32-bit .NET application on Windows 7 x64 that seems to center around a 3rd party COM library. Here's the setup:

Our application was written for 32-bit Windows XP, but we tried to set things up so that it would run properly on 64-bit Windows. It relies on P/Invoke and COM Interop with several 32-bit 3rd party libraries. Obtaining 64-bit versions of these libraries is not possible.

The library in question is a COM wrapper that is provided by the vendor to their C-based framework library (not supported past Windows XP). This interface is far easier to work with from .NET. However, on 64-bit Windows 7, the COM library seems to malfunction during its initialization. The framework has a function named Init() which returns with success, but it does not behave "properly" afterward.

The framework is supposed to look up configuration information in the registry during initialization (names of custom DLLs that we created and other metadata). Using Sysinternals Process Monitor I can see it querying in the 32-bit compatibility section of the registry, but it does not seem to use what it finds.

I am able to reproduce this behavior by interacting with the COM interface from PowerShell (x86) so I do not think I have a setting wrong in Visual Studio.

Here's the kicker though. If I run our application on the x64 platform from Visual Studio with the Hosting Process enabled, it works every time. If I disable the hosting process, or run from outside Visual Studio, it fails every time.

Any ideas what it is about the hosted environment that allows this COM interaction to succeed?

UPDATE: The application is definitely running in 32-bit mode. Visual Studio is configured to build it for x86 and I can see the *32 after its name in Task Manager. Process Monitor also shows the WOW64 DLLs being utilized.

Answer 1

One difference is that the visual studio hosting process does not statically depend on your code. So it gets launched first, .net is loaded and and appdomain is created, and then your .exe is loaded as if it were a dll.

If this is the key difference, then you should be able to run your application outside of the debugger by replicating this scenario. Make a dummy .net exe that does nothing except starting up, and then load a dll dynamically that contains the actual program.

That may brings you a step closer towards understanding/solving the problem, as it rules out other issues, and allows you to proceed from something that works.

Answer 2

You cannot load 32-bit DLL's into a 64-bit process (and vice-versa). If you have an app which consumes a DLL which wraps and loads another DLL, etc., all DLL's in the chain MUST be 64-bit in order to load. If one of the DLL's is 32-bit, they will not load and your app is likely to fail.

Because you're dependent on these 32-bit DLL's it's probably wisest to mark all your EXE's as 32-bit only. You can leave your own DLL's as able to support 32-bit or 64-bit, but by limiting your EXE's to 32-bit only, then you'll force all loaded resources to 32-bit also.

Was there a specific reason you were porting your app to 64-bit?