ret, retn, retf - how to use them

Question

I have the following asm code:

; int __stdcall wWinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPWSTR lpCmdLine, int nShowCmd)
_wWinMain@16 proc near

var_8= dword ptr -8
var_4= dword ptr -4
hInstance= dword ptr  8
hPrevInstance= dword ptr  0Ch
lpCmdLine= dword ptr  10h
nShowCmd= dword ptr  14h

push    ebp
mov     ebp, esp
sub     esp, 8
mov     [ebp+var_4], 5
mov     eax, [ebp+var_4]
add     eax, 1
mov     [ebp+var_8], eax
xor     eax, eax
mov     esp, ebp
pop     ebp
retn    10h

From what I read, you have 3 types of return instruction: ret, retn and retf, meaning return, return near and return far. They allow an optional argument nBytes, that I guess it's the number of bytes to pop, from the defined variables. When should I use retn or retf instead of ret? How can I calculate the optional parameter, nBytes?

Answer 1

There are actually only two different returns, retn and retf. When you just use ret, the assembler or compiler is smart enough to pick which one is necessary. A near return is a jump to within the existing code segment, a far return is a jump to a different code segment. On Windows you only have a single code segment, and thus ret should just be a mnemonic for retn. Separate retn and retf instructions are a throwback to older days when segmented memory models were common. Pretty much all 32-bit x86 systems running today uses a flat, not segmented, memory model.

Ret with no argument pops the return address off of the stack and jumps to it. Some calling conventions (like __stdcall) specify that the callee function cleans up the stack. In this case, they call ret with number of bytes to pop those parameters off of the stack. The 16 bytes are the parameters to the winmain function.

Answer 2

It's actually two types: retn and retf. The third one ret is coded by the assembler into one of the first two.

The difference is that retn (return near) will pop the instruction pointer (IP) only. While the retf (return far) will pop both the instruction pointer (IP) and the code segment (CS).

Answer 3

In the mnemonic ret N, N is the size of parameters on the stack. In this case it is 4 * 4 = 16 (10h) for 4 DWORDs.
But this only applies to calling conventions when the callee is responsible for stack cleanup. In case of cdecl convention the ret should be without any numbers, as the caller is responsible for stack cleanup.