I just ran into an issue where a stack overflow in a threaded c++ program on HPUX caused a SEGV_MAPERR when a local object tried to call a very simple procedure. I was puzzled for a while, but luckily I talked to someone who recognized this as a stack size issue and we were able to fix the problem by increasing the stack size available to the threads.
How can I recognize when the stack overflows? Do the symptoms differ on windows/linux/hpux?
Assuming you're not on a platform thats going to stop your app and say "stack overflow" I suspect you'll see the same behavior that you would see from any kind of buffer overflow. The stack is just another preallocated chunk of memory for your program, and if you go outside those bounds... well good luck! Who knows what you'll stomp on!
You could write over the temperature readout from the CPU, it could be the email you're typing to Larry, it could be the bit saying that the kernel is locked, causing a fun deadlock condition! Who knows.
As for C++, there's nothing saying how the stack should be laid out in relation to other things in memory or that this thing even needs to be a stack!
Exception code 0xC00000FD on Windows.
Usually it's easier to diagnose when you realize your SEH stops working.
Perhaps a bit off topic, but the analagous issue in Ada (running out of stack space in tasks) is a rather common "uncommon" error. Many compilers will stop the task (but not the main task) with a PROGRAM_ERROR exception.
In a way, you almost have to be able to sniff out this one. It tends to start with something like, "I moved this big array inside my task, and suddenly it quit working".
How can I recognize when the stack overflows?
If you know the stack size, where the stack starts and the direction it grows in memory, you can simply check the address of the stack pointer and see if it past the end of the stack. C++ does not allow direct access to the stack pointer. You could easily write a small function in assembly to perform this analysis and link it into you program.