calling code stored in the heap from vc++

Question

Imagine I am doing something like this:

void *p = malloc (1000);
*((char*)p) = some_opcode;
*((char*)p+1) = another_opcode; // for the sake of the example: the opcodes are ok

....
etc...

How can I define a function pointer to call p as if it was a function? (i'm using VC++ 2008 express).

Thanks

Answer 1

If you have the right opcodes in place, calling can be as simple as casting to a function pointer and calling it.

typedef void (*voidFunc)();

char *p = malloc (1000);
p[0] = some_opcode;
p[1] = another_opcode; // for the sake of the example: the opcodes are ok
p[n] = // return opcode...

((voidFunc)p)();

Note though that unless you mark the page as executable, your processor may not let you execute code generated on the heap.

Answer 2

Actually, malloc probably won't cut it. On Windows you probably need to call something like VirtualAlloc in order to get an executable page of memory.

Starting small:

void main(void)
{
    char* p = (char*)VirtualAlloc(NULL, 4096, MEM_COMMIT, PAGE_EXECUTE_READWRITE);
    p[0] = (char)0xC3;  // ret

    typedef void (*functype)();
    functype func = (functype)p;
    (*func)();
}

The next step for playing nice with your code is to preserve the EBP register. This is left as an exercise. :-)

After writing this, I ran it with malloc and it also worked. That may be because I'm running an admin account on Windows 2000 Server. Other versions of Windows may actually need the VirtualAlloc call. Who knows.

Answer 3

A comment wasn't enough space. Joe_Muc is correct. You should not stuff code into memory obtained by malloc or new. You will run into problems if you change the page properties of pages that Windows allocates.

This isn't a problem becuase using VirtualAlloc() and the related WIn32 APIs is every easy: call VirtualAlloc() and set the flProtect to [PAGE_EXECUTE_READWRITE][2]

Note, you should probably do three allocations, one guard page, the pages you need for your code, then another guard page. This will give you a little protection from bad code.

Also wrap calls to your generated code with structured exception handling.

Next, the Windows X86 ABI (calling conventions) are not well documented (I know, I've looked). There is some info here, here, here The best way to see how things work is to look at code generated by the compiler. This is easy to do with the \FA switches ( there are four of them).

You can find the 64-bit calling conventions here.

Also, you can still obtain Microsoft's Macro Assembler MASM here. I recommend writing your machine code in MASM and look at its output, then have your machine code generator do similar things.

Intel's and AMD's processor manuals are good references - get them if you don't have them.