Using DXUTSetWindow in a MFC Direct3D application

Question

I've been experimenting with the DXUT functions for Direct3D applications in the March 2009 DirectX SDK. They seem to have many useful features including auto detection of DirecX10 or DirectX9 and window management features that bypass a lot of the tedious window management tasks generally required by Direct3D. However, I'm having a problem when I use the DXUTSetWindow function to have Direct3D draw to an already created window. In this case, it is the view child window of a CFrameWndEx that is the application's main window.

If I set the bHandleMessages parameter to true, everything resizes and resets without any intervention on my part. But the application crashes when I try to close the program (and immediately exits, so I can't even break to see what is causing the crash) with a bunch of memory leaks. If I set bHandleMessages to false, none of the resizing works, but I don't get a crash nor any memory leaks on exit.

It looks like the DXUTMainLoop is looking for a WM_QUIT message to exit and then clean up all of the Direct3D objects, which is never received by the child window. I've tried the following with no success:

1. Overriding the CFrameWndEx's OnClose function and manually sending a WM_QUIT message to the child.
2. Putting the DXUTMainLoop call on a worker thread.
3. Overriding CFrameWndEx::DefWindowProc and using the DXUTStaticWndProc function (that the DXUT documentation says to use if using DXUTSetWindow, but neglects to show in any of the examples).
4. Setting the window to use the CFrameWndEx's handle and creating a separate swap chain for the child window. It seems that DXUT blacks out the handle that is used in DXUTSetWindow regardless of whether anything is done to it in the OnD3D9FrameRender callback.

UPDATE: After overriding the DefWindowProc of the child window and calling DXUTStaticWndProc from the child window's DefWindowProc, I was able to get the application to run, resize, handle device lost, and exit without the crash, memory leaks. This is done using bHandleMessages set to false. However, this causes a new issue:

The Direct3D drawing surface is a couple pixels smaller than the child window's area, leaving a border of video noise surrounding the drawing surface. This doesn't happen when bHandleMessages is true, even though it is operating on the same HWND in the same DXUTStaticWndProc function.

UPDATE 2: So it looks like the pixel problem is now solved. Using the scheme from the first update to this question, I found I needed to only call CWnd::DefWindowProc in the child window's overridden DefWindowProc only if DXUTStaticWndProc didn't return 0. So the following code seems to solve it:

LRESULT ChildView::DefWindowProc(UINT message, WPARAM wParam, LPARAM lParam){

    LRESULT lr = DXUTStaticWndProc(this->GetSafeHwnd(), message, wParam, lParam);
    if(lr != 0) {
     return CWnd::DefWindowProcW(message, wParam, lParam);
    }

    return lr;
}

My other question still stands though, is it even worth using the DXUT library? How much will it limit me in what I can do using DirectX? Should I just go back to managing all Direct3D objects, device setup/reset and rendering manually?

Answer 1

In my opinion, the DXUT library is junk. I would just steal whatever code you need from it and insert that directly into your application and apply refactorings liberally. The whole DXUT philosophy seems to be "even though its 2009, people think objects are hard, so we're going to set the wayback machine for 1988 and code everything C style with global functions". Its just awful.

Answer 2

Thanks man! You just saved my life for a quick mockup! Works perfectly.