Ah, so many questions ;-)
I'm trying to use the stencil buffer to "mask out" pixels when rendering a full screen quad, but it doesn't seem to work.
First, I fill the stencil buffer like this:
m_renderDevice->SetState(STATE_ZENABLE, 1);
m_renderDevice->SetState(STATE_ZWRITE_ENABLE, 1);
m_renderDevice->SetState(STATE_STENCIL_ENABLE, 0);
m_renderDevice->SetState(STATE_STENCIL_FUNC, COMPARE_ALWAYS);
m_renderDevice->SetState(STATE_STENCIL_PASS, STENCIL_REPLACE);
m_renderDevice->SetState(STATE_STENCIL_FAIL, STENCIL_REPLACE);
m_renderDevice->SetState(STATE_STENCIL_ZFAIL, STENCIL_REPLACE);
m_renderDevice->SetState(STATE_STENCIL_REF, 1);
m_renderDevice->SetState(STATE_STENCIL_MASK, 0xFF);
// Stencil buffer is cleared to "0" before rendering
RenderScene();
Nevermind the DirectX-wrapper code, everything maps directly to SetRenderState calls.
Now what I think this does is write a "1" to the stencil buffer wherever there's geometry
When I apply this stencil buffer when rendering "regular" geometry, it works as expected:
m_renderDevice->SetState(STATE_ZWRITE_ENABLE, 1);
m_renderDevice->SetState(STATE_STENCIL_ENABLE, 1);
m_renderDevice->SetState(STATE_STENCIL_FUNC, COMPARE_EQUAL);
m_renderDevice->SetState(STATE_STENCIL_PASS, STENCIL_KEEP);
m_renderDevice->SetState(STATE_STENCIL_FAIL, STENCIL_KEEP);
m_renderDevice->SetState(STATE_STENCIL_ZFAIL, STENCIL_KEEP);
m_renderDevice->SetState(STATE_STENCIL_REF, 0);
RenderSkybox();
This will render only the parts of the skybox where no scene geometry was rendered before.
However, applying exactly the same render states when rendering a fullscreen quad results in nothing being rendered.
My fullscreen quad consists of pre-transformed vertices (if that has anything to do with it).
So I was wondering if there's an explanation for this, or if I just screwed up.
Thanks a lot in advance!
