How to authenticate windows application against other application/COM Object

Question

I have 3 components in my system:

1. COM Object - Provide Services to application that has func1(), func2()
2. App1 - Trusted Application that need to use the com object funcs (1 and 2)
3. App2 - Malicious application, not authorized to use func1(), can use func2() it is not harmful.

How can the COM Object can "authenticate" App1 and allowing it to use func1() and func2() and deny access to func1() from App2 ?

One way to do it is by allowing only Administrators users to access func1() but this is not a good solution because of security best practice: run with least privileged user. App1 will only need admin to access to the COM Object, any security hole in App1 will give the attacker Admin access.

How can this be solved?

Answer 1

Wouldn't the same methods for licensing be useful? I.e. some kind of key that you provide to the COM object to unlock its methods?

Answer 2

Windows security is user based so I do not believe that you will be able to do this at the application level. If the user can execute the function, then both programs will be able to execute the function.

Answer 3

In general you should define more exactly how you want to devide (identify) "good" aplication which are allowed to use your COM object from other "bad" applications.

If your COM object are in-proc server (a DLL which will be loaded in the address space of the application which use it) then you can make "quick & dirty" solution: Inside of the DllMain you can test the name of the exe file which loaded your dll. You can do this with respect of GetModuleFileName with NULL as the first parameter. If a "wrong" exe try to load your dll the DllMain can return FALSE. The same test you can do in any of your method instead of DllMain.

The best general way to solve your problem (the best which I see of cause) will be to add an additional method to your COM Object which you can use to authorize the caller. For example, to use any "secret" functions like func1() you can require the caller to call another authorize() function before. The caller give your COM Object as input prameter of authorize() some information which can be used to verify the caller permissions. If the authorization is OK, authorize() will gives back an authorization token (cookie) which can be anything which you can easy to verify later. The best tokens should be based on cryptografical algorithms like digitaly signing. The function func1() can have an additional parameter - the token (cookie) received from authorize1(). In this way you can implement any kind of authorization which you want. This way will works with any kind of COM Objects (not only with in-proc-servers).

Answer 4

I think @quip was referring to the IClassFactory2 set of interfaces that supports licensing. See here:

http://msdn.microsoft.com/en-us/library/ms680095(v=VS.85).aspx

The article touches on per machine licenses (which is not what you want) and runtime license keys which sounds like what you are looking for.

The point is that App1 which is authorized, should call CoGetClassObject() to get an object that implements IClassFactory2, and then call IClassFactory2::CreateInstanceLic() passing in a secret key that lets the COM server know that it is authorized. This will in turn instantiate your COM object with the appropriate flags indicating that is available for full use (assuming a valid key). If an invalid key is passed in, initialize your COM object for use by an unauthorized client.

App2 which is not authorized, will call the standard CoCreateInstance() which under the covers call CoGetClassObject() to get an object that implements IClassFactory, and then call IClassFactory::CreateInstance(). This implementation should instantiate your COM object with flags set for an unauthorized client.