C#: How to test for StackOverflowException

Question

Say you have a method that could potentially get stuck in an endless method-call loop and crash with a StackOverflowException. For example my naive RecursiveSelect method mentioned in this question.

Starting with the .NET Framework version 2.0, a StackOverflowException object cannot be caught by a try-catch block and the corresponding process is terminated by default. Consequently, users are advised to write their code to detect and prevent a stack overflow. For example, if your application depends on recursion, use a counter or a state condition to terminate the recursive loop.

Taking that information (from this answer) into account, since the exception can't be caught, is it even possible to write a test for something like this? Or would a test for this, if that failed, actually break the whole test-suite?

Note: I know I could just try it out and see what happens, but I am more interested in general information about it. Like, would different test frameworks and test runners handle this differently? Should I avoid a test like this even though it might be possible?

Answer 1

How about checking the number of frames on the stack in an assert statement?

const int MaxFrameCount = 100000;
Debug.Assert(new StackTrace().FrameCount < MaxFrameCount);

In your example from the related question this would be (The costly assert statement would be removed in the release build):

public static IEnumerable<T> SelectRecursive<T>(this IEnumerable<T> subjects, Func<T, IEnumerable<T>> selector)
{
    const int MaxFrameCount = 100000;
    Debug.Assert(new StackTrace().FrameCount < MaxFrameCount);

    // Stop if subjects are null or empty
    if(subjects == null || !subjects.Any())
        yield break;

    // For each subject
    foreach(var subject in subjects)
    {
        // Yield it
        yield return subject;

        // Then yield all its decendants
        foreach (var decendant in SelectRecursive(selector(subject), selector))
            yield return decendant;
    }
}

It's not a general test though, as you need to expect it to happen, plus you can only check the frame count and not the actual size of the stack. It is also not possible to check whether another call will exceed stack space, all that you can do is roughly estimate how many calls in total will fit on your stack.

Answer 2

We cannot have a test for StackOverflow because this is the situation when there is no more stack left for allocation, the application would exit automatically in this situation.

Answer 3

You would need to solve the Halting Problem! That would get you rich and famous :)

Answer 4

It's evil but you can spin it up in a new process. Launch the process from the unit test and wait for it to complete and check the result.

Answer 5

The idea is to keep track of how deeply a recursive funcion is nested, so that it doesn't use too much stack space. Example:

string ProcessString(string s, int index) {
   if (index > 1000) return "Too deeply nested";
   s = s.Substring(0, index) + s.Substring(index, 1).ToUpper() + s.Substring(index + 1);
   return ProcessString(s, index + 1);
}

This of course can't totally protect you from stack overflows, as the method can be called with too little stack space left to start with, but it makes sure that the method doesn't singelhandedly cause a stack overflow.

Answer 6

First and foremost I think the method should handle this and make sure it does not recurse too deep.

When this check is done, I think the check should be tested - if anything - by exposing the maximum depth reached and assert it is never larger than the check allows.