How do I prevent and/or handle a StackOverflowException? (C#)

Question

I would like to either prevent or handle a StackOverflowException that I am getting from a call to the XslCompiledTransform.Transform method within an Xsl Editor I am writing. The problem seems to be that the user can write an Xsl script that is infinitely recursive, and it just blows up on the call to the Transform method. (That is, the problem is not just the typical programmatic error, which is usually the cause of such an exception.)

Is there a way to detect and/or limit how many recursions are allowed? Or any other ideas to keep this code from just blowing up on me?

Thanks.

Answer 1

From Microsoft:

Starting with the .NET Framework version 2.0, a StackOverflowException object cannot be caught by a try-catch block and the corresponding process is terminated by default. Consequently, users are advised to write their code to detect and prevent a stack overflow. For example, if your application depends on recursion, use a counter or a state condition to terminate the recursive loop.

I'm assuming the exception is happening within an internal .NET method, and not in your code.

You can do a couple things.

• Write code that checks the xsl for infinite recursion and notifies the user prior to applying a transform (Ugh).
• Load the XslTransform code into a separate process (Hacky, but less work).

You can use the Process class to load the assembly that will apply the transform into a separate process, and alert the user of the failure if it dies, without killing your main app.

EDIT: I just tested, here is how to do it:

MainProcess:

// This is just an example, obviously you'll want to pass args to this.
Process p1 = new Process();
p1.StartInfo.FileName = "ApplyTransform.exe";
p1.StartInfo.UseShellExecute = false;
p1.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;

p1.Start();
p1.WaitForExit();

if (p1.ExitCode == 1)    
   Console.WriteLine("StackOverflow was thrown");

ApplyTransform Process:

class Program
    {
        static void Main(string[] args)
        {
            AppDomain.CurrentDomain.UnhandledException += new UnhandledExceptionEventHandler(CurrentDomain_UnhandledException);
            throw new StackOverflowException();
        }

        // We trap this, we can't save the process, 
        // but we can prevent the "ILLEGAL OPERATION" window 
        static void CurrentDomain_UnhandledException(object sender, UnhandledExceptionEventArgs e)
        {
            if (e.IsTerminating)
            {
                Environment.Exit(1);
            }
        }
    }

Answer 2

I would suggest creating a wrapper around XmlWriter object, so it would count amount of calls to WriteStartElement/WriteEndElement, and if you limit amount of tags to some number (f.e. 100), you would be able to throw a different exception, for example - InvalidOperation.

That should solve the problem in the majority of the cases

Answer 3

With .NET 4.0 You can add the HandleProcessCorruptedStateExceptions attribute from System.Runtime.ExceptionServices to the method containing the try/catch block. This really worked! Maybe not recommended but works.

using System;
using System.Reflection;
using System.Runtime.InteropServices;
using System.Runtime.ExceptionServices;

namespace ExceptionCatching
{
    public class Test
    {
        public void StackOverflow()
        {
            StackOverflow();
        }

        public void CustomException()
        {
            throw new Exception();
        }

        public unsafe void AccessViolation()
        {
            byte b = *(byte*)(8762765876);
        }
    }

    class Program
    {
        [HandleProcessCorruptedStateExceptions]
        static void Main(string[] args)
        {
            Test test = new Test();
            try {
                //test.StackOverflow();
                test.AccessViolation();
                //test.CustomException();
            }
            catch
            {
                Console.WriteLine("Caught.");
            }

            Console.WriteLine("End of program");

        }

    }      
}

Answer 4

If you application depends on 3d-party code (in Xsl-scripts) then you have to decide first do you want to defend from bugs in them or not. If you really want to defend then I think you should execute your logic which prone to external errors in separate AppDomains. Catching StackOverflowException is not good.

Check also this question.