Thought experiment with __stdcall and corrupted stack (C++)

Question

My mind was wandering today on the topic of function pointers, and I came up with the following scenario in my head:

__stdcall int function (int)
{
    return 0;
}

int main()
{
    (*(int(*)(char*,char*))function)("thought", "experiment");
    return 0;
}

AFAIK this code would corrupt the stack, so what types of issues could I be looking at if I ran this code?

I'd do this investigating myself however I'm away from my dev machine for a week.

EDIT: Hold on a second, I've been thinking a bit more. As has been observed in the comments, the intent of this code was to have a parameter left on the stack when all is said and done (caller puts two params on the stack, callee -- expecting only one param -- pops only one off). However, since my cast doesn't make mention of the calling convention, am I casting away stdcall, at least from the view of the caller? int function(int) will still pop a param off the stack, but does the caller revert to thinking the function is __cdecl (the default) because of the cast? (i.e. three total params popped?)

EDIT2: The answer to that second question, as confirmed by Rob, is yes. I would have to restate __stdcall if I wanted to leave a param on the stack:

(*(__stdcall int(*)(char*,char*))function)("thought", "experiment");

Answer 1

I think you would have 'undefined behavior' in this case.

From the C standard: (I would assume it's the same in C++)

768 If a converted pointer is used to call a function whose type is not compatible with the pointed-to type, the behavior is undefined.

Edit: On most operating system, this type of error would not cause problems in your whole operating system. But it would cause undefined problems in your program. It would be very hard for a user mode program to be able to cause a blue-screen.

Answer 2

No, it will definitely not cause a blue screen. No user-mode process is able to do that. Even if such bug were in kernel-mode code, the BSOD would occur only after accessing invalid memory or passing wrong arguments to a function.

You are simply corrupting private memory of your process, and the corruption may (or may not) later result in an invalid operation (eg. dereferencing a pointer pointing to invalid memory). When this happens, the OS terminates your process, but no sooner.

Answer 3

You are calling the function as if it is _cdecl which means the caller pushes the arguments and cleans up the stack.

The receiving function is _stdcall which implies the callee cleans up the stack. The callee is expecting a single argument so will pop 4 bytes off the stack.

When the function returns the caller will then pop off two pointers (having previously pushed on two pointers), so your stack is being corrupted by 4 bytes.

Both calling conventions use the same return mechanism, and have the same register rules (eax, ecx and edx are not preserved). See wikipedia for more details.

Depending on the stack frame layout and alignment this mismatch could cause a number of effects. If you are lucky then you get away with it. If not you might mess up the return address of your main function, causing the program to crash when it branches to who-knows-where. If the compiler has injected some kind of stack guard to catch corruption then it will likely detect this and abort the program.