How to force process isolation for an out of process COM server?

Question

I'm writing managed code that has to interact with a vendor's COM automation server that runs out of process. I have found that this server becomes unstable if more than one client connects to it. For example, if I have managed code in process A and managed code in process B both connecting to this COM server, a single process will be spun up for the COM server and it's behavior is less than reliable. I'm hoping there's a way to force a separate process for each client - server connection. Ideally, I'd like to end up with:

Managed Process A talking to COM Server in process C1
Managed Process B talking to COM Server in process C2

One thought that came to mind was that if I ran process A and process B with different security identities, that that might cause the COM infrastructure to create separate server processes. I'd rather not go down that road, however. Managed Process A and Managed Process B are actually Windows Services. And I'm running them with identity Local System (because I need them to be able to interact with the desktop, and you can't check the "Interact with Desktop" box on the services applet for services that don't run as Local System). And the reason I need to interact with desktop is that this COM server occasionally throws up a dialog box on the screen and if the service itself cannot interact with the desktop then the COM server is spawns can't display the dialog (I believe it is displayed on a hidden WinStation).

Answer 1

It's been a while since I've done this, so my memory is hazy.

If you configure the OOP COM server as a DCOM server using the DCOM config tool, I believe you can specify the isolation level. I did this years ago with a non-threadsafe in-process DLL that needed to be accessed in a threadsafe fashion from IIS and it worked a charm.

Let me know if it works for you :)

Answer 2

Your best bet would be to get the vendor to fix the component. After all, if it won't handle multiple clients, there could be other bugs lurking. If you can't do this, there are some other things to try.

With in-process COM objects I've had occassion to manually load the dll and access the interfaces directly without going through COM.

I haven't done this myself with out-of-process COM, but there are some things you could try. Ultimately the library is just a process sitting there receiving messages which invoke functions.

You might be able to manually start the a new copy of the process for each client and then send it messages. You may run into some hiccups with this. For example, the process may check to see if it's already running and refuse to start or otherwise be unhappy.

If you have a known upper limit on the number of clients, another approach you could consider would be to make multiple copies of the original .exe file and then use binary patching (something similar to the detours library from Microsoft Research) to override the COM registration functions and register each copy as a separate COM object.

Answer 3

Place the component registered at COM+, this put an isolation layer at your.
Use : Control Panel->Administrative Tools
or cmd/execute DCOMCNFG

Component Services->Computers->My Computer->COM+ Application, right click, new application, next, Create an empty application, enter app name “COM+ your.dll”, next, select Local Service, next, next, next, finish.

In new item made, expand, at Components, right click, new component, next, select Install new component, select your component.

Click Component properties, tab Identity, select System Account.

For errors in calls see Event after.