[ebp + 6] instead of +8 in a JIT compiler

Question

I'm implementing a simplistic JIT compiler in a VM I'm writing for fun (mostly to learn more about language design) and I'm getting some weird behavior, maybe someone can tell me why.

First I define a JIT "prototype" both for C and C++:

#ifdef __cplusplus 
    typedef void* (*_JIT_METHOD) (...);
#else
    typedef (*_JIT_METHOD) ();
#endif

I have a compile() function that will compile stuff into ASM and stick it somewhere in memory:

void* compile (void* something)
{
    // grab some memory
    unsigned char* buffer = (unsigned char*) malloc (1024);

    // xor eax, eax
    // inc eax
    // inc eax
    // inc eax
    // ret -> eax should be 3
    /* WORKS!
    buffer[0] = 0x67;
    buffer[1] = 0x31;
    buffer[2] = 0xC0;
    buffer[3] = 0x67;
    buffer[4] = 0x40;
    buffer[5] = 0x67;
    buffer[6] = 0x40;
    buffer[7] = 0x67;
    buffer[8] = 0x40;
    buffer[9] = 0xC3; */

    // xor eax, eax
    // mov eax, 9
    // ret 4 -> eax should be 9
    /* WORKS!
    buffer[0] = 0x67;
    buffer[1] = 0x31;
    buffer[2] = 0xC0;
    buffer[3] = 0x67;
    buffer[4] = 0xB8;
    buffer[5] = 0x09;
    buffer[6] = 0x00;
    buffer[7] = 0x00;
    buffer[8] = 0x00;
    buffer[9] = 0xC3; */


    // push ebp
    // mov ebp, esp
    // mov eax, [ebp + 6] ; wtf? shouldn't this be [ebp + 8]!?
    // mov esp, ebp
    // pop ebp
    // ret -> eax should be the first value sent to the function
    /* WORKS! */
    buffer[0] = 0x66;
    buffer[1] = 0x55;
    buffer[2] = 0x66;
    buffer[3] = 0x89;
    buffer[4] = 0xE5;
    buffer[5] = 0x66;
    buffer[6] = 0x66;
    buffer[7] = 0x8B;
    buffer[8] = 0x45;
    buffer[9] = 0x06;
    buffer[10] = 0x66;
    buffer[11] = 0x89;
    buffer[12] = 0xEC;
    buffer[13] = 0x66;
    buffer[14] = 0x5D;
    buffer[15] = 0xC3;

    // mov eax, 5
    // add eax, ecx
    // ret -> eax should be 50
    /* WORKS!
    buffer[0] = 0x67;
    buffer[1] = 0xB8;
    buffer[2] = 0x05;
    buffer[3] = 0x00;
    buffer[4] = 0x00;
    buffer[5] = 0x00;
    buffer[6] = 0x66;
    buffer[7] = 0x01;
    buffer[8] = 0xC8;
    buffer[9] = 0xC3; */

    return buffer;
}

And finally I have the main chunk of the program:

void main (int argc, char **args)
{
    DWORD oldProtect = (DWORD) NULL;
    int i = 667, j = 1, k = 5, l = 0;

    // generate some arbitrary function
    _JIT_METHOD someFunc = (_JIT_METHOD) compile(NULL);

    // windows only
#if defined _WIN64 || defined _WIN32
    // set memory permissions and flush CPU code cache
    VirtualProtect(someFunc,1024,PAGE_EXECUTE_READWRITE, &oldProtect);  
    FlushInstructionCache(GetCurrentProcess(), someFunc, 1024);
#endif

    // this asm just for some debugging/testing purposes
    __asm mov ecx, i

    // run compiled function (from wherever *someFunc is pointing to)
    l = (int)someFunc(i, k);

    // did it work?
    printf("result: %d", l);

    free (someFunc);
    _getch();
}

As you can see, the compile() function has a couple of tests I ran to make sure I get expected results, and pretty much everything works but I have a question...

On most tutorials or documentation resources, to get the first value of a function passed (in the case of ints) you do [ebp+8], the second [ebp+12] and so forth. For some reason, I have to do [ebp+6] then [ebp+10] and so forth. Could anyone tell me why?

Answer 1

Your problem is the 66 and 67 bytes -- operand size override and address size override, respectively.

Since you're running this code in 32-bit mode, these bytes tell the processor that you want 16-bit operands and addresses instead of 32-bit ones. The 66 55 disassembles to PUSH BP, which pushes only 2 bytes instead of 4, hence your addresses being off by 2.

The 67 bytes in the first two examples are also unncessary, but because you're only accessing registers and not memory, they have no effect and don't break anything (yet). Those bytes should also be removed.

It looks like you're using a framework designed for 16-bit code, or perhaps there's a way you can tell it you want 32-bit code.

Answer 2

Your opcodes look suspicious: they're full of 0x66 and 0x67 address/data size override prefixes, which (in a 32-bit code segment) will turn 32-bit operations into 16-bit ones. e.g.

buffer[0] = 0x66;
buffer[1] = 0x55;
buffer[2] = 0x66;
buffer[3] = 0x89;
buffer[4] = 0xE5;
...

is

push bp
mov  bp, sp

rather than

push ebp
mov  ebp, esp

(which seems to explain the observed behaviour: pushing bp decrements the stack pointer by 2 instead of 4).