Disable Background drawing in JFrame in order to properly display Aero (DWM) effects.

Question

I'm having problems using the DWM functionality of Windows Vista/7 on Java windows. I want to make the background of my frame use the Aero style. The Windows API to do so is provide by the function DwmExtendFrameIntoClientArea in the dwmapi library. I've managed to call the procedure properly via JNA, and it does what it is supposed to do (You can see that for example when resizing the frame, before the next repaint you see the proper aero effects in the area not yet painted, see the attached image).

But somewhere (I can't figure out where) a background is painted over the Aero effect and the effect is lost.

What I have already tried:

• Using a custom ContentPane with opacity set to false
• Setting the opacity of the LayeredPane and the RootPane to false
• Using a Frame instead of a JFrame
• Set the background color of the JFrame/ContentPane to black/fully transparent
• Use setLayersOpaque and a custom variant thereof, see first answer for more details

So far I could not succeed removing that background. Is it a limitation of AWT/Swing? How can I remove that background or use the Aero effect properly?

Your help is greatly appreciated.

Screenshot

Here a screenshot of a frame without any contents, having set the opacity of the RootPane, LayeredPane and ContentPane to false. I did it quickly while resizing. You see that the effect is properly applied to the area Java did not yet paint on.

http://i55.tinypic.com/v614qo.png (As a new user I cannot post the image directly...)

Odd behavior

Upon further investigation I came across the following odd behavior. If the window size is 150x150 or below the contents are displayed transparently. This is very glitchy for normal window components. If you paint directly on the frame by overriding the paint() method everything is drawn semi-transparent. Additionally the coordinate system seems to be a little off, it appears as the zero point of the JFrame is set to the actual zero point of the window. Thus Swing tries to paint to areas where actually the window border is located, which then of course is not visible.

See this screenshot: http://d-gfx.kognetwork.ch/java_aero_bug.png

Example code

This is the code I use.

Requires jna.jar and platform.jar. Available from the JNA homepage.

import com.sun.jna.Function;
import com.sun.jna.Native;
import com.sun.jna.NativeLibrary;
import com.sun.jna.Structure;
import com.sun.jna.platform.win32.WinDef.HWND;
import com.sun.jna.platform.win32.WinNT.HRESULT;
import javax.swing.JFrame;
import javax.swing.JLabel;
import javax.swing.UIManager;

public class AeroFrame extends JFrame {

    public AeroFrame() {
        setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

        JLabel label = new JLabel("Testlabel");
        label.setOpaque(false);

        add(label);

        pack();

        enableAeroEffect();
    }

    private void enableAeroEffect() {
        NativeLibrary dwmapi = NativeLibrary.getInstance("dwmapi");
        HWND aeroFrameHWND = new HWND(Native.getWindowPointer(this));
        MARGINS margins = new MARGINS();
        margins.cxLeftWidth = -1;
        margins.cxRightWidth = -1;
        margins.cyBottomHeight = -1;
        margins.cyTopHeight = -1;
        //DwmExtendFrameIntoClientArea(HWND hWnd, MARGINS *pMarInset)
        //http://msdn.microsoft.com/en-us/library/aa969512%28v=VS.85%29.aspx
        Function extendFrameIntoClientArea = dwmapi.getFunction("DwmExtendFrameIntoClientArea");
        HRESULT result = (HRESULT) extendFrameIntoClientArea.invoke(HRESULT.class,
                new Object[] { aeroFrameHWND, margins});
        if(result.intValue()!=0)
            System.err.println("Call to DwmExtendFrameIntoClientArea failed.");
    }

    /**
     * http://msdn.microsoft.com/en-us/library/bb773244%28v=VS.85%29.aspx
     */
    public class MARGINS extends Structure implements Structure.ByReference {
            public int cxLeftWidth;
            public int cxRightWidth;
            public int cyTopHeight;
            public int cyBottomHeight;
    }

    public static void main(String[] args) {
        try {
            UIManager.setLookAndFeel(UIManager.getSystemLookAndFeelClassName());
            JFrame.setDefaultLookAndFeelDecorated(true);

        } catch (Exception e) {
            e.printStackTrace();
        }
        new AeroFrame().setVisible(true);
    }

}