overflow technique in stack

Question

int main(void) {
   problem2();
}

void doit2(void) {
    int overflowme[16];
    //overflowme[37] =0;
}

void problem2(void) {
    int x = 42;
    doit2();
    printf("x is %d\n", x);
    printf("the address of x is 0x%x\n", &x);
}

Would someone help me understand why overflowme[37] =0; from the doit2 function will overwrite the value of x? (please include Program Counter and Frame Pointer of the function doit2 in your explanation) Thank you!

It works every time on a x86 windows machine (ok there!) with Project properties->Configuration properties->C/C++ ->Code Generation->Basic Runtime Checks set to "Default". so it's not an undefined behavior.

Answer 1

It may not. The location of variables in the stack is compiler- and platform-dependent.

Answer 2

It doesn't:

x is 42
the address of x is 0xbff9ea1c

The above happens every time on the One True Compiler and Platform (address changes), so clearly you're right that it's not undefined behavior.

Answer 3

Your stack is going to look something like this:

char overflowme[16]
return address to problem2() from calling doit2()
parameters for doit2(), if it had any
int x = 42
return address to main() from calling problem2()
parameters for problem2(), if it had any
local variables for main(), if it had any

When you write to overflowme[37], you're going to go past the end of overflowme (since it's only 16 bytes) and past the return address from the doit2() call and overwrite x.

As others have mentioned, this is highly dependent on the platform and compiler, but should give you a good visualization of the problem. Try stepping through your code with a debug window open and showing you the stack.

Answer 4

Like everyone else has said, this is dependent of the target and the compliler, but for you those are remaining constant and there aren't any other things in that code that look like they are introducing randomness to the stack (relatively speaking), so it will do the same thing every time.

The system stack typically grows from a high address down to lower addresses. If the stack pointer is 0x1234 and you push a value (on a 32 bit {4 byte} system) then the stack pointer will become 0x1230.

Arrays are addressed from lowest address to highest address. If you have

char a[2];

and a[0] is at 0x0122 then a[1] will be at 0x0123.

Your array in doit2 is an automatic variable, meaning that it is created upon entry to the function and deleted upon exit of the function. Automatic variables have to either live on the stack or in registers. Since it is an array it is much less complicated for the compiler to put it in RAM rather than registers (this makes indexing it easier because it just adds the index*size to the address of the first member of the array). Since the stack is in RAM the compiler puts the array on the stack.

Allocating space on the stack for this array means that the stack pointer is sizeof(int)*16 less than it would be if this array were not present. The stack pointer most likely points to overflowme[0] while in doit2.

There are other things that could be on the stack and a couple things that had to be on the stack. The things that had to be on the stack are the return pointers which were pushed there when the functions were called. On a 32 bit system these should take up 4 bytes each. The things that could have been on the stack (if the compiler wanted to use it) is the previous frame pointer. (Explicit*) Stack frames on x86-32 are just the space between ESP and EBP, but they aren't necessary so often they aren't used and EBP is just used as a general purpose register instead (more general purpose registers available is generally good). Using stack frames is useful, though, because they make debugging much easier because ESP and EBP act as markers for the edges of the local variables. Stack frames are necessary sometimes, such as when you use alloca or C99's variable sized automatic arrays because they allow the local variable space for a function to be discarded by mov EPB, ESP or equivalent instructiosn rather than sub size_of_local_variable, ESP so the compiler doesn't have to know the size of the frame. They also allow the local variables to be addressed relative to EBP rather than ESP which in the case of alloca changes. EBP in this case would not change until the end of the current function unless it was changed and restored by calling functions.

When compiling without optimizations turned on compilers often always use stack frames due to their making debugging easier. It is also easier to model the code with stack frames and then transforming the code to not use them after proving that they aren't necessary.

So, the previous value of EBP may or may not reside on the stack between the return address (somewhere in problem2) and the last element in doit2's overflowme. The compiler is also free to put anything else on the stack that it feels like, so who knows what else might be there.

problem2's local variable int x could go in either a register or on the stack. When compiling without optimizations turned on local variables often go on the stack even when they could go into registers.

So, lets assume that there is doit2's overflowme array, an old frame pointer, a return address, and problem2's x on the stack (and some more stuff under {which is really at a higher address} it).

Since &(overflowme[i]) is the same as the adding the address of the first element of overflowme to (i* {the size of int} ) and the old EBP lies after the last element of overflowme and a return address lies after the old EBP and int x lies after the return address, x is definitely standing right in the way to be run over by a buffer overrun.

Why this happens for the index of 37 is not clear. The pointer math (assuming only the items I stated above being on the stack between the array and x) doesn't suggest that it should be based on 4 byte pointers (32 bit machine), though if this is an 8 byte pointer system (64 bit machine) then the math is closer to the address I'd expect x to be at if sizeof(int) == 8. The compiler also could have gone ahead and allocated stack space for the calls to printf (the variable arguments after the format string must go on the stack) which would effect the math (and also encourage the compiler to place x on the stack because it would have to push it there anyway).

If you want a more detailed answer to your question look at the assembly for this code and work out the exact addressing.

• You could consider that the stack frame is there even if EBP isn't used as the frame Base Pointer, but then the frame wouldn't be framed.

Answer 5

You are very lucky that it only clobbered x. Often times that kind of code can make demons fly out of your nose!