Problem in process hooking

Question

I have a process (say, for example, MyProcessA), hooked an exe and injected my dll (MyDll.dll) into the process space of MyProcessA, so even if it's gonna create n number of child processes it will be process hooked as well. I have no problem in hooking and injecting the dll into the process. I have hooked all file and process dependant functions, but somehow I am not able to achieve complete hook of any setup (any application setup). I suspect if am missing any process related APIs or it might be some UAC problem, currently I am using CreateProcess(A&W), NtCreateProcess, ShellExecute(A&W). What could be the problem?

Answer 1

I suspect that the answer is related to the "Windows Installer Service". I'm guessing that your hooks wouldn't catch any interactions with a service, which even if launched as a result of FireFox's setup is going to be created by a different System process. I haven't had much experience with Windows Installer, but the documentation here should have more details than you could possibly wish for, given the time to find it.

UAC might also cause you issues, but you should be able to rule that out by launching the hooking code with administrative privileges to start with.

Is this research for uni? Either way good luck, it sounds like an interesting problem.