JIT compilation and DEP

Question

Hello. I was thinking of trying my hand at some jit compilataion (just for the sake of learning) and it would be nice to have it work cross platform since I run all the major three at home (windows, os x, linux). With that in mind, I want to know if there is any way to get out of using the virtual memory windows functions to allocate memory with execution permissions. Would be nice to just use malloc or new and point the processor at such a block.

Any tips?

Answer 1

One possibility is to make it a requirement that Windows installations running your program be either configured for DEP AlwaysOff (bad idea) or DEP OptOut (better idea).

This can be configured (under WinXp SP2+ and Win2k3 SP1+ at least) by changing the boot.ini file to have the setting:

/noexecute=OptOut

and then configuring your individual program to opt out by choosing (under XP):

Start button
    Control Panel
        System
            Advanced tab
                Performance Settings button
                    Data Execution Prevention tab

This should allow you to execute code from within your program that's created on the fly in malloc() blocks.

Keep in mind that this makes your program more susceptible to attacks that DEP was meant to prevent.

It looks like this is also possible in Windows 2008 with the command:

bcdedit.exe /set {current} nx OpOut