32bit to 64bit inline assembly porting

Question

I have a piece of C++ code (compiled with g++ under a GNU/Linux environment) that load a function pointer (how it does that doesn't matter), pushes some arguments onto the stack with some inline assembly and then calls that function, the code is like :

unsigned long stack[] = { 1, 23, 33, 43 };

/* save all the registers and the stack pointer */
unsigned long esp;
asm __volatile__ ( "pusha" );
asm __volatile__ ( "mov %%esp, %0" :"=m" (esp));

for( i = 0; i < sizeof(stack); i++ ){
    unsigned long val = stack[i];
    asm __volatile__ ( "push %0" :: "m"(val) );
}

unsigned long ret = function_pointer();

/* restore registers and stack pointer */
asm __volatile__ ( "mov %0, %%esp" :: "m" (esp) );
asm __volatile__ ( "popa" );

I'd like to add some sort of

#ifdef _LP64
   // 64bit inline assembly
#else
   // 32bit version as above example
#endif

But i don't know inline assembly for 64bit machines, anyone could help me?

Thanks

Answer 1

Not really answering your question, but I think you might be able to achieve this in a platform independent way by use of setcontext (or makecontext).

Answer 2

While it shouldn't be much of a problem to call a function pointer with the appropriate arguments in inline assembly, I don't think recoding this naively in x64 will help you, because the calling conventions to be used are very probably different (defaults for 32bit and 64bit linux are definitely different). Have a look here for details. So I guess, if you can get away without inline assembly in this case (see the other answer), it'll be easier to port.

Edit: OK, I see you may have to use assembly. Here are some pointers.

According to Agner Fog's document, linux x64 uses RDI, RSI, RDX, RCX, R8, R9 and XMM0-XMM7 for parameter transfer. This implies that in order to achieve what you want (disregarding floating-point use) your function will have to:

(1) save all registers that need to be saved (RBX, RBP, R12-R15): Set aside space on the stack and move these registers there. This will be somthing along the lines of (Intel syntax):

sub rsp, 0xSomeNumber1
mov [rsp+i*8], r# ; insert appropriate i for each register r# to be moved

(2) Evaluate the number of arguments you will have to pass by stack to the target function. Use this to set aside the required space on the stack (sub rsp, 0xSomeNumber2), taking into account 0xSomeNumber1 so that the stack will be 16-byte aligned at the end, i.e. rsp must be a multiple of 16. Don't modify rsp after this until your called function has returned.

(3) Load your function arguments on the stack (if necessary) and in the registers used for parameter transfer. In my view, it's easiest if you start with the stack parameters and load register parameters last.

;loop over stack parameters - something like this
mov rax, qword ptr [AddrOfFirstStackParam + 8*NumberOfStackParam]
mov [rsp + OffsetToFirstStackParam + 8*NumberOfStackParam], rax

Depending on how you set up your routine, the offset to the first stack parameter etc. may be unnceccessary. Then set up the number of register-passed arguments (skipping those you don't need):

mov r9, Param6
mov r8, Param5
mov rcx, Param4
mov rdx, Param3
mov rsi, Param2
mov rdi, Param1

(4) Call the target function using a different register from the above:

call qword ptr [r#] ; assuming register r# contains the address of the target function

(5) Restore the saved registers and restore rsp to the value it had on entry to your function. If necessary, copy the called function's return value wherever you want to have them. That's all.

Note: the above sketch does not take account of floating point values to be passed in XMM registers, but the same principles apply. Disclaimer: I have done something similar on Win64, but never on Linux, so there may be some details I am overlooking. Read well, write your code carefully and test well.

Answer 3

Main problems:

• pushad/popad is missing on x64, you have to push the individual registers you wish to save
• you need to save your rsp (64 bit stack pointer) in a suitable 64 bit register (rax?, r8? etc)
• calling convention has almost certainly changed from 32 bit to 64 bit

Summary of changes from x86 to x64:

• Registers starting with E now have 64 bit equivalents starting with R. RAX, RBX, RCX, RDX, RDI, RSI, RIP, RSP, RBP.
• New registers: R8 ... R15
• No pushad, No popad

I've ported some inline x86 code to x64 on Windows. You should definitely take some time to read the x64 instruction set and also to read the calling convention for your operating system. The change on Windows was radical and the new calling convention a lot simpler. I suspect the GNU/Linux change will also be different, I would definitely not assume it is the same.

I would agree with a previous answer that if you can use an alternative method than coding natively, do so. In my case, I couldn't avoid it.