I'm having real trouble getting my stencil test to work correctly with a basic deferred shading implementation. I'm aware there are other options, but I really need to get the stencil test working because I plan to try and use stencil volume shadows. I've done pretty much this exact implementation before, and it worked fine, so I'm really at a loss for what the problem is.
In any case, the real problem is that the stencil test doesn't seem to properly take account of the first pass depth buffer, even though depth test is on during the stencil pass. In addition, the stencil test seems corrupted when it's behind the geometry. Three screenshots below and then stencil and depth buffer grab.
In this first image, the light volume that I'm rendering to the stencil buffer and then again in the light pass is behind terrain geometry. So why is it visible?
This second image looks alright from the other side, although you can see underneath the geometry a bit again.
The third image shows the output of the stencil past, so clearly the geometry is ok.
Here's the rendering code:
// geometry pass m_gbuffer->bindWrite(); // binds fbo m_gbuffer->setDrawBuffers(0, m_num_drawbuffers); // glDrawBuffers(num_buffers, buffers) glEnable(GL_DEPTH_TEST); glDepthMask(GL_TRUE); glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT); glEnable(GL_CULL_FACE); updateCameraInformation(m_camera_information); world->geometryRender(m_camera_information); glDepthMask(GL_FALSE); // light pass glDrawBuffer(GL_COLOR_ATTACHMENT0 + m_num_drawbuffers); // same fbo, so same depth buffer, but different texture for light pass m_gbuffer->bindToTextures(); // bind fbo textures for shaders glEnable(GL_STENCIL_TEST); glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT); // stencil pass glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE); glEnable(GL_DEPTH_TEST); glDisable(GL_CULL_FACE); glStencilFunc(GL_ALWAYS, 0, 0); glStencilOpSeparate(GL_BACK, GL_KEEP, GL_INCR, GL_KEEP); glStencilOpSeparate(GL_FRONT, GL_KEEP, GL_DECR, GL_KEEP); m_point_light.render(); glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE); // light pass glStencilFunc(GL_NOTEQUAL, 0, 0xFF); glDisable(GL_DEPTH_TEST); glEnable(GL_BLEND); glBlendEquation(GL_FUNC_ADD); glBlendFunc(GL_ONE, GL_ONE); glEnable(GL_CULL_FACE); glCullFace(GL_FRONT); m_point_light.render(); glCullFace(GL_BACK); glDisable(GL_STENCIL_TEST); glDisable(GL_CULL_FACE); m_ambient_light.render(); glDisable(GL_BLEND);
Thanks for your time. I appreciate any suggestions.
I swear, posting on these forums just makes me think differently and I solve it easily. The glStencilOpSeparate needs to be set to GL_INCR_WRAP or GL_DECR_WRAP respectively. The WRAP portion means that decrementing at 0 will wrap the buffer to 255, and incrementing at 255 will wrap the buffer to 0. Very useful, yet my previous code that worked did not use WRAP. It must be somewhat platform dependent. Hope this can help someone.
glStencilOpSeparate(GL_BACK, GL_KEEP, GL_INCR_WRAP, GL_KEEP); glStencilOpSeparate(GL_FRONT, GL_KEEP, GL_DECR_WRAP, GL_KEEP);