OpenGL - Question about the usage of glDepthMask

Question

Hello all,

I have rendered an objectA in a scene as follows. The scene has many other objects too.

void Draw()
{    
    if( glIsList( displayListID ) )
    {
        glPushAttrib( GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT|GL_ENABLE_BIT );

        glEnable( GL_BLEND );
        glEnable( GL_DEPTH_TEST );
        //glDepthMask( GL_FALSE );
        glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );

        glEnable( GL_LINE_SMOOTH );
        glEnable( GL_POINT_SMOOTH );
        glEnable( GL_POLYGON_SMOOTH );

        glMatrixMode( GL_MODELVIEW ); 

        color.setAlpha(alpha); // set alpha transparent of this objectA
        glCallList( displayListID );

        //glDepthMask( GL_TRUE );
        glDisable( GL_BLEND );  

        glPopAttrib();
    }
}

Now here is the problem,

As shown, I comment out two lines //glDepthMask( GL_FALSE ); //glDepthMask( GL_TRUE );

the scene renders the objectA and other objects correctly in depth. However, the modification of the objectA alpha doesn't work anymore (i.e. color.setAlpha(alpha) ).

If I uncomment the above two lines, then alpha modification is back to work. However, the depth rendering is NOT correct. In other words, sometimes, the objectA should be behind other objects but it looks like the objectA is in front of all objects.

How do I fix this problem?

Thank you

Answer 1

One possible solution: set glDepthMask to GL_TRUE at all times and draw all your non-transparent objects first (in any order, just as you seem to do it now), then draw all your (semi-)transparent objects sorted from back to front.

In some cases, if you don't care about the order in which your (semi-)transparent objects are drawn and you only want the other, opaque objects to "shine through", you can skip sorting your (semi-)transparent objects.

Answer 2

1. Turn on the depth mask glDepthMask( GL_TRUE )
2. Draw all opaque objects, in any order
3. Turn off the depth mask glDepthMask( GL_FALSE )
4. Draw translucent objects sorted from furthest away to nearest

Why do you do this?

There are 2 buffers you need to worry about: the depth buffer and the color buffer. These buffers are really just big 2d arrays, each the width x height of your screen.

The color buffer naturally is going to hold the final coloration of each pixel. There is one entry in the color buffer per screen pixel. The depth buffer, is like the color buffer in that there is one entry per screen pixel, but it is used for something different. Entries in the depth buffer are a measure of "how close" each colored pixel really is.

If you render 1 triangle, that is far away from the camera, it generates a set of colors and depth values for each pixel it "wants" to cover on the screen. Say you then another poly that is closer, it also will generate a set of values for the depth and color buffers. Now, there is a sort of "contest" at pixel coloration time where the "further away" fragments (large depth buffer values) are discarded, and only the closest fragments are kept. The closer fragments end up coloring the pixel you had. (When two polygons are nearly overlapping, Z-fighting can occur)

Start by rendering the objects in your scene with the depth mask on. This means every shape you render, when its pixels get colored, the depth buffer gets updated with the "winner" of the contest.

Then, you 3) glDepthMask( GL_FALSE ) turns off the depth buffer for writing, 4) render translucent shapes from furthest to nearest. Seems weird, huh?

When you turn off the depth mask, and render the translucent shapes, OpenGL will still read the depth buffer to determine which fragments to throw away (i.e. if your translucent shape is behind an already rendered solid shape, then you throw that translucent shape's fragments away). But it will not write to the depth buffer, so if a translucent shape is really really close to the eye (say like a translucent windshield), those fragments do not prevent other fragments that are actually further away from being drawn. This is important, because if your windshield is right in front of you and you render it translucent, and you let the windshield fragments update the depth buffer then you will see nothing else in your scene except the windshield, even though there are shapes behind it, because OpenGL will think "Hey, that windshield is the only thing the user should see, due to these depth buffer readings, so I won't bother rendering anything further away than this windshield, then." Turning off the depth mask is a way of "tricking" OpenGL into "not knowing" there are very close, but translucent, fragments.