Catching exit(1);

Question

I have a MFC SDI application which during startup loads a DLL. I am only able to view the source code and use the DLL but not changing & recompiling it.

The situation now is that, whenever the DLL encouner an error it will call exit() such as below.

bool Func()
{
  // .. do something here

  if (error) { exit(999); }
}

In my MFC application, i've set the SetUnhandledExceptionFilter to handle all exceptions and also created a MiniDump for debugging purposes.

So now the problem is that whenever the DLL encounter any error, it will just call exit() with the status code 999 and my ExceptionFilter will not catch it and thus no MiniDump created for PostMortem debugging.

I was wonder if:
1. Is there any other way for my global exception handler to catch this?
2. Can i override the exit() function so that when its called, i call do this "throw("error encounter!")" and my global exception handler can catch it.
3. I tried using atexit() at my MFC application, whereby i register another function to throw error whenever the DLL call exit(). But it seems like this is not working as well.

What i really wanted to do is that whenever the DLL encounters an error, i want a MiniDump to be generated so i could do PostMortem debugging. Are there anything else that might work in this situation?

Thanks.

Answer 1

Here's a macro I wrote to place breakpoints in the exit() functions:

Imports System.IO

' Sets breakpoints on all exit functions.  useful for catching library code that 
' calls exit in the debugger.
Sub AddBreakpointsToExit()
    Dim bp As EnvDTE.Breakpoint
    Dim bps As EnvDTE.Breakpoints

    Dim envVar As String = "VS90COMNTOOLS"
    Dim comnTools As String = System.Environment.GetEnvironmentVariable(envVar)
    If (String.IsNullOrEmpty(comnTools)) Then
        Throw New System.Exception("Environment variable '" + envVar + "' doesn't exist.")
    End If
    Dim filePath As String = System.IO.Path.Combine(comnTools, "..\..\VC\crt\src\crt0dat.c")

    ' set exit function names and line #s:
    Dim exitFunctions(0 To 4) As String
    exitFunctions(0) = "exit"
    exitFunctions(1) = "_exit"
    exitFunctions(2) = "_cexit"
    exitFunctions(3) = "_c_exit"

    ' line numbers are based on the Visual Studio 2008 definition.
    ' TODO: check and add options if 2005 or 2010 are differen.t
    Dim exitLines(0 To 4) As Integer
    exitLines(0) = 412
    exitLines(1) = 420
    exitLines(2) = 427
    exitLines(3) = 434

    ' set breakpoints:
    For i = 0 To 3 Step 1
        bps = DTE.Debugger.Breakpoints.Add(File:=filePath, Line:=exitLines(i))
    Next i

End Sub

Answer 2

I haven't yet found a solution for this, but here's one possible reason why your atexit handler isn't helping: the dll may be statically linking to the CRT. This would mean that the code for exit() is built directly in to the dll, which would give it its own private list of exit handlers, so the atexit handler registered from within your host process would never be seen. If you were able to call atexit() from within the dll (which would be pretty tricky to arrange), it would likely work.

Just a guess - hope that helps.

Answer 3

Check the DLL's symbol import table.

You just might be able to remap it to a function of your choosing at runtime.

How to do it (assuming exit is there):

Most DLLs use symbol import via a jump table. If you set a breakpoint on the caller you will see that it calls to a long jump instruction. The jump table is a PAGE_EXECUTE_READWRITE page so you may simply overwrite that slot of the jump table with your own address.

The location of the jump table will vary, but it will always be a constant offset from the load address, and therefore a constant offset from any function pointer into the DLL found by GetProcAddress.

Of course, all this assumes the DLL never changes. There's probably a way to find all this stuff at runtime but I don't know how you'd go about it.

Oh, you cannot return from exit(). You'll have to longjump out.

Answer 4

When the process exits normally, every DLL receives a DLL_PROCESS_DETACH notification. How about writing your own DLL with the exclusive purpose of catching this notification?