Internet Explorer crashes when MSXML2::IXMLDOMDocumentPtr -> Release() is called

Question

I am creating a shell extension in C++ (ATL 9) using Visual Studio 2008. The Shell Extension creates a global MSXML2::IXMLDOMDocumentPtr object m_XmlDoc in the module class. This m_XmlDoc is then used in the extension by all classes to read xml document.

The problem that I am facing is with Internet explorer. When the Shell Extension is active and I open/close internet explorer, I get a debug dialog and IE crashes. The error message says "Unhandled exception at 0x6aac30f1 in iexplore.exe: 0xC0000005: Access violation reading location 0x03050970." When I click "break" on the message window, It takes me to the "Release" method of COM Smart Pointer and the error seems to be on m_pInterface->Release();

This call was made from Module's destructor and also the value of m_pInterface is not NULL. I think maybe internet explorer is using the XML DOM and the call to Release creates some problem in it.

MSXML2::IXMLDOMDocumentPtr m_XmlDoc;

In _AtlModule.Init() method
    ::CoInitialize(NULL);
    m_XmlDoc.CreateInstance(MSXML2::CLSID_DOMDocument40);

dllMain code:

extern "C" BOOL WINAPI DllMain(HINSTANCE hInstance, DWORD dwReason, LPVOID lpReserved)
{
    ::CoInitialize(NULL);
    if (dwReason == DLL_PROCESS_ATTACH)
    {
     _AtlModule.Init(); 
     CreateImageLists();
     ::DisableThreadLibraryCalls(hInstance);
    }

    hInstance;
    return _AtlModule.DllMain(dwReason, lpReserved); 
}

Answer 1

the use of DisableThreadLibraryCalls is discouraged, you seen it?

Answer 2

The problem was because of the COM Smart Pointer used for XmlDomDocument. I changed it to a normal pointer and it is working fine even in Vista.

This problem has a different behaviour in XP and Vista. In XP, I was getting an unhandled exception when I closed Internet Explorer. In Vista, I was not able to browse the virtual drive.

Answer 3

There are at least two problems with your code as posted:

1. You're calling CoInitialize in DllMain.
2. You're creating a COM object in DllMain.
3. It wouldn't surprise me if you're doing something in CreateImageLists() which you also shouldn't be doing in DllMain.

Also, the reason that your crash was "fixed" by not using the smart pointer is because now you're not actually releasing the object anymore. Your code is broken, and not releasing the reference isn't a valid way to fix anything.

I would suggest that you read, and then re-read, the documentation for DllMain paying particular attention to the things you should never do within your implementation of the function. As you'll see right up front:

Warning There are serious limits on what you can do in a DLL entry point. To provide more complex initialization, create an initialization routine for the DLL. You can require applications to call the initialization routine before calling any other routines in the DLL.

I suspect once you read it, and fix your code to create the COM object at a valid time, and release it at a valid time, your shell extension will stop crashing.