How is application virtualization implemented?

Question

I am trying to understand how software like App-V and sandboxie (http://www.sandboxie.com/) work. But for the life of me, I can't think of anything that could make this possible. How do they intercept API calls and trick the target software? If someone would say that it's just magic and pixie dust, I would believe them. Seriously though, are there any white papers that discuss solutions to this problem?

If this is possible on the CLR level then that would be good but I'm willing to go native if I have to.

Answer 1

Sandboxie does it by essentially injecting code into core Windows API, the same way a virus would (which is why Vista x64 prevents this behaviour, and why Sandboxie doesn't work on that OS).

Here is a project explaining API hooking. I learned how all this work by studying the sourcecode for Metamod:Source (used for SourceMod for CounterStrike:Source :) )

Answer 2

I don't know how MS did it, but here is the basic theory of one way to do it ...

What you want to do is hook into the system calls (similar to chaining into interrupt).

1. System call occurs.
2. Your custom intercept gets executed.
3. If this syscall does not need special processing, continue on. Otherwise it needs special processing and go to step 4.
4. Get the stack pointer, instruction pointer and all that jazz from the stack, and build a new stack frame to send you back to your custom code in user-land.
5. Do your massaging of data and paths and stuff in user land. This way if the underlying OS changes, this code does not have to be updated [as frequently].
6. After all the data massaging, execute the system call again.
7. Your custom interrupt executes again, but it should detect that you are calling from your user-land helper layer and pass the call on through. Some stack frame manipulation may be required to set up proper return addresses.
8. Regular system call executes.
9. When the system call returns, the stack frame should should send you back to your regular program flow.

Hope this helps.

Answer 3

Check out the Wikipedia page on X86 Virtualization which discusses both software virtualization (early VMWare, Wine, Sandboxie and to an extent App-V) and the more modern hardware virtualization (Hyper-V, VMWare, others).

I'm assuming you're looking specifically for software virtualization as by using .NET (or any CLR) you're already abstracting yourself away from the CPU architecture to an extent, especially with the 'AnyCPU' target.