Catching a StackOverflowException

Question

How do I catch a StackOverflowException?

I have a program that allows the user to write scripts, and when running arbitrary user-code I may get a StackOverflowException. The piece running user code is obviously surrounded with a try-catch, but stack overflows are uncatchable under normal circumstances.

I've looked around and this is the most informative answer I could find, but still led me to a dead end; from an article in the BCL team's blog I found that I should use RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup to call the code and the delegate that would get called even after a stack overflow, but when trying, the process gets terminated with the stack overflow message without the delegate ever getting called. I've tried adding PrePrepareMethodAttribute on the handler method but that didn't change anything.

I've also tried using an AppDomain and handling both the UnhandledException and the DomainUnload event - but the entire process gets killed on stack overflows. The same happens even if I throw new StackOverflowException(); manually and not get an actual stack overflow.

Answer 1

You need to run the code in a separate process.

Answer 2

To handle an exception that is not handled by your code, you can subscribe to the AppDomains UnhandledException -- which is what the operating system handles when it displays the dialog that says the program exited unexpectedly.

In the Main method of your program use

var currentDomain = AppDomain.CurrentDomain;

and then add a handler to the event

currentDomain.UnhandledException += handler;

In the handler you can do anything you want, such as log, display an error, or even reinitializing the program if desired.

Answer 3

Program your script engine to trace the level of recursion in the script. If the recursion goes above some arbitrarily large number then kill the script before it kills your program. Alternatively you could program the script engine to operate in a stackless manner and store all of the script's stack data in a System.Collections.Generic.Stack<T>. Even if you do use a separate stack you will still want to limit the level of recursion that a script can have, but stack collection will give you a few hundred times more stack space.

Answer 4

You must load the user script, or any external 3rd party plugin, in a different app domain, so that you can safely unload the domain should an unrecoverable error occurs.

You must create a different AppDomain since you cannot unload an assembly from a loaded domain, and you don't want to shutdown your main application domain.

You create a new application domain like this:

var scriptDomain = AppDomain.CreateDomain("User Scripts");

You can then load any type from an assembly that you need to create. You have to be sure that the object that you will load inherits from MarshalByRefObject.

I assume that your user script is wrapped inside an object defined like this:

public abstract UserScriptBase : MarshaByRefObject
{
    public abstract void Execute();
}

You can therefore load any user script like this:

object script = domain.CreateInstanceFromAndUnwrap(type.Location, type.FullName);

After all that, you can subscribe to the scriptDomain.UnhandledException and monitor any unrecoverable error.

Using a different application domain is not easy and you will most likely encounter some loading/unloading problem (DLL is referenced by both domain).

I recommend that you fellow some tutorial that you could find online.