ansaurus

Question

Why does stack overflow throw no error in Visual C++?

Answer 1

+5 A:

A stack overflow is undefined behaviour. The compiler is well within it's rights to ignore it or cause any event to happen.

DeadMG 2010-08-04 21:23:22

Thanks. But how are you meant to catch a stack overflow event in Visual c++ (or in general if possible)?

qaz 2010-08-04 21:29:52

@qaz: You're not. In Visual C++ 2010, you CAN catch Structured Exceptions (SEH) from Windows, which include things like access violation, stack overflow and such. However, the language is not designed or intended to catch such events.

DeadMG 2010-08-04 21:41:38

Visual C++ is most certainly designed to catch such events, that's why it has language extensions and compiler support (`__try` / `__except`)

Ben Voigt 2010-08-05 00:19:33

Visual C++ is. C++ itself isn't.

DeadMG 2010-08-05 09:23:07

Answer 2

+8 A:

C++ won't hold your hand as a managed enviroment does. Having a stack overflow means undefined behaviour.

Tamás Szelei 2010-08-04 21:24:22

Thanks. But how are you meant to catch a stack overflow event in Visual c++ (or in general if possible)?

qaz 2010-08-04 21:29:15

@qaz You are supposed to write code that does not cause a stack overflow in the first place.

anon 2010-08-04 21:33:07

@Neil I agree, but what if you are doing some recursion based on user input. How do you know if the user has input something which will cause a stack overflow? Do you keep a running count of how much memory you've allocated on the stack? It seemed to me that a natural way to handle this was if an error was thrown when too much memory has been allocated just as the new operator does.

qaz 2010-08-04 21:39:15

@qaz:You don't. You structure your program to avoid such events. Cleaning up after a stack overflow is impossible to implement without a significant performance penalty throughout your program. The lack of such checks is the reason why C and C++ code can be so fast when compared to other, more managed languages.

slacker 2010-08-04 21:40:12

@qaz Unless your recursive function is badly designed or inappropriate, you won't have a stack overflow. And yes, if you are dealing with user input (say brace nesting, for example) you need to set a limit at which your app bales out in your code.

anon 2010-08-04 21:46:58

@Billy ONeal:Try to use SEH to intercept fatal errors (such as SO), and continue after them, when mixed with C++ features like exceptions, destructors, etc. and soon you find yourself in burning **HELL**.

slacker 2010-08-04 21:53:22

Is there away to handle stack overflows without using SEH? For example what if you're using the g++ compiler?

qaz 2010-08-04 22:16:35

@qaz:There is no way to handle stack overflows in C and C++ - with or without SEH. Even if you catch the SEH exception thrown when you overflow the stack (or you use any other platform-specific mechanism to detect the SO), at this point your program is basically already totally screwed. Trying to continue doing *anything* can unleash nasal demons on you.

slacker 2010-08-04 22:25:21

@slacker This is what I feared. I like the idea of recursion, it makes for simpler code, but not being able to effectively handle stack overflows puts me off using recursion on the stack. Instead I end up writing my own stack on the heap to do the same job, if I run out of memory at least I get a bad_alloc exception. However writing recursion on the heap is messy so in general it means I avoid recursion :(

qaz 2010-08-04 22:46:11

@slacker: Unless your system throws that stack overflow notice before you've actually overflown the stack -- in which case you can throw a C++ exception which will unwind the stack correctly. It's the detection of stack overflow though which is platform specific. See boost::regex's source code for an example.

Billy ONeal 2010-08-04 23:07:41

@qaz I and most other people have been using recursion successfully for all our long careers, without having to write special code. You should in no way avoid using recursion, or implement your own stacks to support it. You can avoid stack overflow by checking function parameters before you call recursive functions, or during the recursive function call itself, but this is rarely necessary. For example, when you call qsort(), do you worry about its stack usage - you shouldn't.

anon 2010-08-04 23:24:49

@qaz:You do realize that heap allocations are expensive, right? An easier and more efficient way to make recursion "overflow-proof" is to track recursion depth explicitly. You add one more parameter to your function: `max_recursion_depth` (or whatever you want to call it). At the beginning of the function you decrement it, and if it gets negative you handle the "overflow" error however you want. Then, when you initially call the function you need to pass some tested, safe value for maximum depth, which you know doesn't blow the stack up.

slacker 2010-08-04 23:26:32

@Neil Butterworth:You do know that `qsort()` is these days always implemented with some stack-safe and performance-safe algorithm (never quick sort!), right?

slacker 2010-08-04 23:34:06

@slacker I meant to say "worry about recursion". But qsort is often implemented with a quicksort variant - introsort. See http://en.wikipedia.org/wiki/Introsort

anon 2010-08-04 23:41:31

@slacker Yeah I realize it's expensive, but it guarantees overflow proof code as you said.Often I can allocate all the memory I need on the stack (avoiding slow heap allocation). It seems I can make at most 10,000 nested function calls on the (default) stack before overflow, but if I only need to allocate one int per function call I can search to a depth of at least 100,000 by implementing my own stack (which is allocated on the system's stack).

qaz 2010-08-04 23:51:45

@slacker I don't particularly like the idea of setting a fixed depth limit. It's a bit like setting a maximum array size and never checking if new throws a bad_alloc. Then again implementing your own stack also has its downside. Hence my reason for making this post.

qaz 2010-08-04 23:52:39

@Neil Butterworth:Interesting - thanks for the link! Still, it would be more accurate to call it a mixed algorithm - not a variant.

slacker 2010-08-04 23:56:03

@qaz: butbutbut... your heap has a maximum limit as well, somewhere! there are always limits. if you're calling a function 10,000 times recursively, you're almost always doing something you could do better in another way. There are clearly lots of experienced people here to help and explain-- why not post your algorithm so we can discuss that?

quixoto 2010-08-05 00:00:22

@qaz as hard as it is to fathom, computers actually DO have limits. If you want to have a larger stack for your program, then give it one. There are linker/build options that let you determine the stack size for your program. Also, if you write your code to be tail recursive, and many algorithms can be written this way, the compiler may very well optimize the stack out of your code completely (letting you have an infinite search depth). You'll need to check your compiler for details.

Will Hartung 2010-08-05 00:04:57

@quixoto I have no particular algorithm at the minute. I solved my problems by implementing my own stack on the heap. I was writing a program that was part of a maths proof. If it failed due to memory issues that was fine but it should say so. If it appeared to terminate successfully you could have (some) certainty in it's answer. The code would be clearer using recursion (clarity was more important than speed or memory effeciency) however a stack overflow could make it look like the program succeeded where it actually failed.

qaz 2010-08-05 00:07:11

@quixoto my issue isn't that there are memory limitations, but I expected you should be able to squeeze the most of it much like the heap. I don't like the idea of just setting a limit and hoping that under all circumstances this will stop overflow from happening. I imagine there could be situations where the bound should be theoretically set very low, but if you could keep track of the amount allocated in practice you could search much further than the worst case scenario.

qaz 2010-08-05 00:11:02

Detecting stack overflow is both possible and safe. Recovering from it is not, but reporting it and/or starting a crash dump is, if you take appropriate precautions (i.e. there's another thread, with unused stack, that actually writes out the dump file, or even better another process, so you don't have to worry about loader lock issues). All your signal handler (and I'm including SEH exception filters in that category) should do is unpause another task (and I'm using that to mean either separate thread or separate process).

Ben Voigt 2010-08-05 00:18:39

@Will Hurtung: I agree that increasing the stack is an option which will increase the search depth plus not cause expensive heap operations. However, as I explained to quixoto, my main concern was detecting stack overflows which is easily caught by a bad_alloc exception if you implement your own stack. Also the code was meant for math academics, and although I can expect (some of) them to take the source code and compile it, it's unreasonable to expect that they'll be bothered enough to find out how to increase the stack size option on their compilers.

qaz 2010-08-05 00:21:58

@qaz: OK, fair enough. But I will note that maths proofs represent an extremely narrow use case for programming. I've never, in a decade of professional software development, including on embedded devices, seen a "legitimate" stack overflow (ie, one that wasn't caused by a bug/logic error). This is just not something that is normally worried about, because very few code designs lean on stack recursion to anywhere near that sort of depth. It sounds like your use case is special, in which case, yeah, build what you need to build to support it. But do know that it's unusual in general.

quixoto 2010-08-05 00:23:01

@Ben Voigt: Thanks. I was hoping there was a simple generic way you could detect stack overflows and recover from them. I guess there isn't.

qaz 2010-08-05 00:27:26

@quixoto Yeah I agree with you completely. It's very unusual, and when I'm programming non-math proof related things I never worry about it. However, this website is great at answering my unusual weird questions, plus I usually learn something from understanding what happens in these extreme situations and how to handle them.

qaz 2010-08-05 00:32:45

@Neil Butterworth: Sorry missed your comment first time around. Yeah I do worry about stack overflow in qsort (I implemented my own, though speed wasn't an issue so I could've implemented a slower sorting algorithm) I like every eventuality to be covered as best as I can. I often would like to write clear code which searches a large space via recursion. However, the default limit of 10000 nested calls is far too small. Often I can spot stack overflow has occured, and choose to implement my own stack. However I assumed there would be some error thrown or displayed when it happened.

qaz 2010-08-05 01:12:50

I'm a little surprised this isn't a more common problem. For example when dealing with Tree based data structures, it's easy to imagine a lopsided tree with over 10000 nodes down one side could occur. Recursion seems like a good way to iterate through the tree, but stack overflow will occur. You may not know beforehand it will end up lopsided but I would've thought you would want to test you're iteration didn't cause an overflow which is a possibility.

qaz 2010-08-05 01:20:54

@qaz: re: your surprise that it's so hard to detect/handle overflows: it's because so few programs in practice require this kind of stack depth. Architectures are designed with little practical allowance for recovery/scoping/sizing of that kind of out of bounds because the bounds are usually orders of magnitude larger than anyone uses, so you end up in a real pickle if you do get there. FWIW, tree traversals can sometimes be implemented iteratively within a single function, more or less, with the right data structure. (http://en.wikipedia.org/wiki/Tree_traversal)

quixoto 2010-08-05 02:24:04

(Or keep your trees balanced better. ;) )

quixoto 2010-08-05 02:25:50

re: standard method for detection... there is, of course, but it isn't part of the C or C++ standard. Signal processing is part of the POSIX specification which is ~99% of all operating systems. Unfortunately the one OS that doesn't implement this is -- you guessed it -- Windows. So you're essentially left needing two alternate implementations -- catching SIGILL (ILL_STK) and SIGSEGV will cover all POSIX OSes (use the SA_ONSTACK option when registering your handler) and an SEH filter for EXCEPTION_STACK_OVERFLOW on Windows -- together this covers pretty much all environments with hard MMU.

Ben Voigt 2010-08-05 03:34:06

@Ben: Yes, Windows doesn't support that. OTOH POSIX doesn't allow you to throw a C++ exception from inside a signal handler to unwind the stack and continue program execution.

Billy ONeal 2010-08-06 13:59:10

@Billy: There's a lot of information here, although I can't quite decode whether C89 does or does not permit `longjmp` from signal handler. http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1318.htm

Ben Voigt 2010-08-06 14:53:18

@Ben: C89 doesn't say anything about signal handlers -- you'd have to look in POSIX instead. In any case, longjmp is problematic in C++ because it doesn't call destructors on the stack. (At least according to § 18.10.4 in the latest draft C++ standard: `A setjmp/longjmp call pair has undefined behavior if replacing the setjmp and longjmp by catchand throw would invoke any non-trivial destructors for any automatic objects.` )

Billy ONeal 2010-08-06 16:47:44

@Billy: There's a catch-22 here. Either `throw` out of a signal handler is allowed to misbehave, or `longjmp` produces undefined behavior, but not both?

Ben Voigt 2010-08-06 17:01:20

Answer 3

+1 A:

In debugging mode, you want the overhead. You want it to detect if you broke your stack, overflowed your buffers, etc. This overhead is built into the debug instrumentation, and the debugger. In high level terms the debug instrumentation is extra code and data, put there to help flag errors, and the debugger is there to detect the flagged errors and notify the user (in addition to helping you debug, of course).

If you are running with the project compiled in release mode, or without a debugger attached, there is no one to hear the screaming of your program when it dies :) If a tree falls in the forest...

Depending on how you program, C++ is programming without training wheels. If you hit a wall, no one is going to be there to tell you that you screwed up. You will just crash and burn, or even worse, crash and keep running along in a very crippled state, without knowing anything is wrong. Because of this, it can be very fast. There are no extra checks or safe guards to keep it from blazing through your program with the full speed and potential of the processor (and, of course, how many extra steps you coded into your program).

Merlyn Morgan-Graham 2010-08-04 21:31:09

The runtime overhead for detecting stack overflow is positively insignificant. You get it for free with any OS that automatically commits memory only when used.

Ben Voigt 2010-08-05 00:14:17

@Ben Voigt: Okay, good point, assuming that is true for every hardware architecture, OS, and implementation of C/C++. My point is that perf is the reason most memory operations are not checked in C, and C++ takes most of that baggage with it.

Merlyn Morgan-Graham 2010-08-05 00:24:03

Well, let's say it's either really really cheap (running with enabled MMU) or so doggone expensive (no MMU, or MMU disabled) that it's not even practical to check in a debug build.

Ben Voigt 2010-08-05 03:18:56

Answer 4

+2 A:

It might well be that the compiler optimized the intended stack overflow away. Consider the following pseudo-code example:

void RecursiveMethod(int n)
{
    if (n % 1024 == 0)
        print n;

    // call recursively
    RecursiveMethod(n + 1);
}

The method will call itself recursively and overflow the stack pretty quickly because there is no exit condition.

However, most compilers use tail recursion, a technique which will transfer the recursive function call into a loop construct.

It should be noted that with tail recursion, the above program would run in an endless loop and not exit silently.

Bart de Smet has a nice blog article where explains how this technique works in .NET:

The Case of The Failed Demo – StackOverflowException on x64

0xA3 2010-08-04 21:32:44

@0xA3, qaz: One reason this may be possible: http://en.wikipedia.org/wiki/Tail_recursion

Merlyn Morgan-Graham 2010-08-04 21:34:11

@qaz Does the program exit after having done its job or does it just exit at the point where the stack overflow should have occured?

Alexandre Jasmin 2010-08-04 21:41:12

This was the example I was using to test out the stack overflow handling. Which explains why no error was thrown.However, I'm still concerned that if C++ won't tell you when stack overflow occurs how are you meant to detect it yourself?

qaz 2010-08-04 21:44:39

@Alexandre It works up to the point where recursion begins, hangs for a bit (while the stack gets eaten up I suppose) then terminates failing to execute the commands after the recursion (unsuprisingly).

qaz 2010-08-04 21:46:34

@qaz: In case your application terminates unexpectedly, e.g. because of a stack overflow you will get a notification from Windows about that, even in release build ("someprogram.exe has stopped working. A problem caused the program to stop working correctly..."). It sounds to me that your program just exits regularly.

0xA3 2010-08-04 21:57:18

@qaz:You probably have optimization turned off, then. With optimization enabled this code should run in constant stack space (and be equivalent to an endless loop).

slacker 2010-08-04 21:57:49

@0xA3 Nope. Ran a Debug build and got no error messages, which is what disturbs me (the fact that a program can appear to have ran successfully but actually it crashed).

qaz 2010-08-04 22:03:18

@slacker Yes I ran without optimizations. My question now is in a more complicated program how would I catch a stack overflow before it happened. I assumed an error would be thrown like the new command, but this isn't required to happen. So its up to the programmer to detect when a stack overflow is about to happen. How would you go about it?

qaz 2010-08-04 22:07:06

@qaz: That surprises me. I tried running a sample program using a slightly modified version of the above sample code (so that the compiler does not apply tail recursion). I got a notification from Windows in both debug and release build with no debugger attached. I'm on Win7 but this notification should be there in Vista as well.

0xA3 2010-08-04 22:07:47

@qaz: The compiler actually helps you finding possible endless recursions. The sample above causes the following warning: "Warning 1 warning C4717: 'RecursiveMethod' : recursive on all control paths, function will cause runtime stack overflow"

0xA3 2010-08-04 22:09:25

@0xA3 I'm on XP, what was your modification?

qaz 2010-08-04 22:11:04

@0xA3 Yeah I get that compiler warning, I managed to make it go away by just adding an extra return which will never be reached. e.g if (n==-1) return;

qaz 2010-08-04 22:14:07

@qaz: "which is what disturbs me (the fact that a program can appear to have ran successfully but actually it crashed)". Ignoring debug/non-debug, this is what C++ does. There are plenty of tools, plenty of static analysis you can do on your code (e.g. compiler warnings), plenty of features in the debugger. Those aren't perfect. Sometimes in C/C++, your program can go into completely undefined behavior/corrupt memory all over the place, but not crash. To quote the original developer of C++, "In C++ it's harder to shoot yourself in the foot, but when you do, you blow off your whole leg"

Merlyn Morgan-Graham 2010-08-05 00:01:27

@Merlyn Morgan-Graham: You're right, I got complacent I guess. I expected the os/c++ spec to make more of a fuss when something as evil as stack overflowing happened. It does detect an overflow and shutdown the process, why it couldn't tell me it did so I have no idea, instead of letting me believe it terminated successfully.

qaz 2010-08-05 00:45:27

Answer 5

+1 A:

In a debug build, a number of stack checks are put in place to help you detect problems such as stack overflows, stack corruption etc. They are not present in release builds because they would affect the performance of the application. As others have pointed out, a stack overflow is undefined behaviour, so the compiler is not required to implement such stack checks at all.

When you're in a debugging environment, the runtime checks will help you detect problems that will also occur in your release build, therefore, if you fix all the problems detected in your debug build, then they should also be fixed in your release build . . . in theory. In practice, sometimes bugs that you see in your debug build are not present in your release build or vice versa.

Stack overflows shouldn't happen. Generally, stack overflows occur only by unintentional recursive function calling, or by allocating a large enough buffer on the stack. The former is obviously a bug, and the latter should use the heap instead.

dreamlax 2010-08-04 21:36:23

Answer 6

+4 A:

Because when your process stack overflows, it is no longer a valid process. Displaying an error message requires a stack.

Raymond Chen went over this recently.

As for why the debugger is able to throw such an exception, in that case the process is being kept around because it's attached in debugging mode to the debugger's process. Your process isn't displaying the error, the debugger is.

On Windows machines, you can catch the SEH exception which corresponds to a stack overflow. For an example, you can see boost::regex's source code (Google for BOOST_REGEX_HAS_MS_STACK_GUARD).

Billy ONeal 2010-08-04 21:40:34

You **CANNOT** extend the stack over the pre-set limit in any way, period. When you start a thread (including the main thread), the system reserves a continuous chunk of address space for the stack. The memory gets automatically committed as it is used, but if you hit the limit, you're screwed. There's no place to extend the stack into, as the stack needs to be continuous in memory (i.e. you can't split it to smaller chunks).

slacker 2010-08-04 22:11:38

The guard page technique described in the article is the standard way any sane system (Windows, Unix, whatever) uses to implement a stack with automatic committing. This does not change the fact that the *address space* for a stack needs to be pre-reserved in its entirety, and cannot be extended afterwards.

slacker 2010-08-04 22:16:25

@slacker: The reason I assumed that was possible was because of BOOST_REGEX_HAS_MS_STACK_GUARD, which allows boost to safely handle the stack overflow exception on Windows. However, it "Safely Handles" it by throwing an exception (doh!). I have removed that section of my answer.

Billy ONeal 2010-08-04 23:04:19

ansaurus

tags:

views:

answers:

Why does stack overflow throw no error in Visual C++?

related questions