How to detect possible / potential stack overflow problems in a c / c++ program?

Question

Is there a standard way to see how much stack space your app has and what the highest watermark for stack usage is during a run?

Also in the dreaded case of actual overflow what happens?

Does it crash, trigger an exception or signal? Is there a standard or is it different on all systems and compilers?

I'm looking specifically for Windows, Linux and Macintosh.

Answer 1

On Windows a stack overflow exception will be generated.

The following windows code illustrates this:

#include <stdio.h>
#include <windows.h>

void StackOverFlow()
{
  CONTEXT context;

  // we are interested control registers
  context.ContextFlags = CONTEXT_CONTROL;

  // get the details
  GetThreadContext(GetCurrentThread(), &context);

  // print the stack pointer
  printf("Esp: %X\n", context.Esp);

  // this will eventually overflow the stack
  StackOverFlow();
}

DWORD ExceptionFilter(EXCEPTION_POINTERS *pointers, DWORD dwException)
{
  return EXCEPTION_EXECUTE_HANDLER;
}

void main()
{
  CONTEXT context;

  // we are interested control registers
  context.ContextFlags = CONTEXT_CONTROL;

  // get the details
  GetThreadContext(GetCurrentThread(), &context);

  // print the stack pointer
  printf("Esp: %X\n", context.Esp);

  __try
  {
    // cause a stack overflow
    StackOverFlow();
  }
  __except(ExceptionFilter(GetExceptionInformation(), GetExceptionCode()))
  {
    printf("\n****** ExceptionFilter fired ******\n");
  }
}

When this exe is run the following output is generated:

Esp: 12FC4C
Esp: 12F96C
Esp: 12F68C
.....
Esp: 33D8C
Esp: 33AAC
Esp: 337CC

****** ExceptionFilter fired ******

Answer 2

gcc places an extra block of memory between the return address and the normal variables in "unsafe" function calls, like (in this example the function is void test() {char a[10]; b[20]}:

call stack:
-----------
return address
dummy
char b[10]
char a[20]

If the function write 36 bytes in the pointer 'a', the overflow will 'corrupt' the return address (possible security breach). But it will also change the value of the 'dummy', that is between the pointer and the return address, so the program will crash with a warning (you can disable this with a -fno-stack-protector)

Answer 3

Stack overflow is probably the nastiest type of exception to handle -- because your exception handler has to deal with a minimal amount of stack (usually only a single page is reserved for this purpose).

For an interesting discussion of the difficulties handling this type of exception see these blog posts: 1 and 2 from Chris Brumme which focus on the issue from the .NET perspective, particularly hosting the CLR.

Answer 4

Some compilers support stackavail() function, which returns the amount of remaining free space of the stack. You can use this function before calling functions in your programs that require a lot of stack space, to determine if it is safe to call them

Answer 5

On Linux you get a segmentation fault if your code tries to write past the stack.

The size of the stack is a properties inherited between processes. If you can read or modify in the the shell using commands like ulimit -s (in sh, ksh, zsh) or limit stacksize (tcsh, zsh).

From a program the size of the stack can be read using

#include <sys/resource.h>
#include <stdio.h>

struct rlimit l;
getrlimit(RLIMIT_STACK, &l);
printf("stack_size = %d\n", l.rlim_cur);

I don't know of a standard way to get the size of the available stack.

The stack starts with argc followed by the contents of argv and a copy of the environment, and then your variable. However because the kernel can randomize the location of the start of the stack, and there can be some dummy values above argc, it would be wrong to assume that you have l.rlim_cur bytes available below &argc.

One way to retrieve the exact location of the stack is to look at the file /proc/1234/maps (where 1234 is the process ID of your program). Once you know these bounds you can compute how much of your stack is used by looking at the address of the latest local variable.

Answer 6

I would suggest you to use alternate-signal-stack if you are on linux.

1. In this case all the signal will be handled over alternate stack.
2. In case stack overflow occurs, system generates a SEGV signal, this can be handled over alternate stack.
3. If you do not use it ... then you may not be able to handle the signal, and your program may crash without any handling/erro-reporting.

Answer 7

On Linux, the Gnu libsigsegv library includes the function stackoverflow_install_handler, which can detect (and in some cases help you recover from) stack overflow.