DWM Composition toggle causes client area to lose alpha

Question

I have a simple Windows application here:

http://www.bengoodger.com/software/chrome/dwm/app.cc

My app provides a customized glass frame for when DWM compositing is active, and a fully custom frame when it is inactive or not available.

The "customized glass frame" consists of a enlarged title bar area, which is reported by my implementation of WM_NCCALCSIZE to be part of the client area since I would like to render transparent controls into it. So that this "tall title bar area" portion of my window is transparent, I fill it with transparent black (BLACK_BRUSH in the simple example above) which causes it to be drawn as glass by the DWM.

When the system DWM is toggled, e.g. by using the Appearance Settings control panel to switch to Vista Basic or Windows Standard, or when an app that requires the system to disable DWM is launched, my app switches to fully custom rendering mode by handling WM_NCPAINT etc. When I switch back, I would expect the reverse, which mostly happens, except my "tall title bar area" is now solid black.

I've found I can work around this issue by getting the window's placement, hiding the window then setting the window's placement again in my WM_DWMCOMPOSITIONCHANGED handling, but this causes other horrible bugs (least of which is window z-order munging).

My question is - what am I doing wrong here? It seems like the window is being put into a bogus state somehow, and hiding/showing it corrects it. How can I prevent this from happening? Any guidance would be greatly appreciated.

Note: I have narrowed this down somewhat. When DWM glass is in effect, any part of the client area that is painted black that is rendered over the client area is rendered transparent. We noticed that when returning to Glass from non-Glass, the client area is rendered solid black instead of transparent. However when the window is maximized and then restored, the window returns to being transparent. When I subsequently drag size the window smaller, the top portion window remains transparent. When I drag it larger, the top portion of the window turns black again. It's as if the DWM is caching the pixels behind the window and sizing the window larger causes it not to be able to paint anything there because its cache isn't large enough. I can't seem to find any DWM function to reset this state. It seems that Maximize/Minimize then Restore or SetWindowPlacement are capable of tickling it though but have other undesirable side effects.

Answer 1

http://www.codeproject.com/KB/dialog/rtaGlassEffectLib.aspx

As you may have noticed, the GlassEnabled property is a great one for testing if this feature is enabled by your system and then start using our library and call ShowEffect() function to show the glass effect, Now suppose the user has disabled/enabled the Aero Theme while your application was running !! This may cause a problem in the appearance of your application.

To solve this problem, two events were added to the library to keep track of the changes happening to the system while your application is running, these two events are GlassEffectEnabled event and GlassEffectDisabled.

Now for people who like to know how stuff works .. I can tell that these events were implemented by monitoring the messages coming from the system to the windows to our application looking for the WM_SYSCOLORCHANGE message that indicates that the system colors have been changed. So by doing a quick compare between the state of the GlassEnabled property before and after this message, we can decide whether this feature has been enabled or not.