How to reliably replace a library-defined error handler with my own?

Question

On certain error cases ATL invokes AtlThrow() which is implemented as ATL::AtlThrowImpl() which in turn throws CAtlException. The latter is not very good - CAtlException is not even derived from std::exception and also we use our own exceptions hierarchy and now we will have to catch CAtlException separately here and there which is lots of extra code and error-prone.

Looks like it is possible to replace ATL::AtlThrowImpl() with my own handler - define _ATL_CUSTOM_THROW and define AtlThrow() to be the custom handler before including atlbase.h - and ATL will call the custom handler.

Not so easy. Some of ATL code is not in sources - it comes compiled as a library - either static or dynamic. We use the static - atls.lib. And... it is compiled in such way that it has ATL::ThrowImpl() inside and some code calling it. I used a static analysis tool - it clearly shows that there're paths on which the old default handler is called.

To ensure I even tried to "reimplement" ATL::AtlThrowImpl() in my code. Now the linker says it sees two declarations of ATL::AtlThrowImpl() which I suppose confirms that there's another implementation that can be called by some code.

How can I handle this? How do I replace the default handler completely and ensure that the default handler is never called?

Answer 1

I encountered similar problems when writing my own memory manager and wanted to overrule malloc and free.

The problem is that ATL::AtlThrowImpl is probably part of a source file that also includes other files that are really needed.

If the linker sees a reference to a function in one of the object files, it pulls in the object file with that function, including all other functions in that same object file.

The solution is to look up in the ATL sources where ATL::AtlThrowImpl is defined and see if the source contains other functions. You will also need to implement the other functions to prevent the linker from having any reference to the original ATL source.

Answer 2

I have not worked these windows libraries in quite a while, but you should be able to get the linker to pick your replacement by putting it in a static library listed earlier in the link statement. Make sure you are using the same method signature as the original. You can also play with linker flags to omit default libraries, for various definitions of default.

If you have a friend with access to MSDN, it may have some of the ATL debug sources, which also helps.

And, as Patrick says, you need to provide a replacement for everything in the same linkage scope. Some libraries break the individual methods out into separate object files to make it easier to replace one piece. That's more common with e.g. c standard lib than for a class. If there's a lot of ATL code capable of calling their throw impl in a single object file, it may be more pain than it is worth....you may need to try catching and rethrowing as your own type.