# **SOLVED** Help with applying stencil buffer to a problem

This topic is 3707 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

I'm currently attempting to use the stencil buffer to correctly clip nested rectangles when drawn to the screen. Here is a screenshot of what I want to occur. So the order is this:
1. Clear stencil buffer to 1
2. Draw red square where stencil buffer ONLY equals 1, and replace with 2
3. Draw green square where stencil buffer ONLY equals 2, and replace with 3
4. Draw yellow square where stencil buffer ONLY equals 3, and replace with 4
5. Draw blue square where stencil buffer ONLY equals 1, and replace with 5
6. Draw light blue square where stencil buffer ONLY equals 5, and replace with 6
The problems I'm having come into play in step 6. The blue square isn't shown. The way I'm setting the stencil state values is as so: glStencilFunc(GL_EQUAL, stencilValue, stencilValue); glStencilOp(GL_KEEP, GL_KEEP, GL_INCR); stencilValue is the value in the stencil buffer where drawing is allowed. So when drawing the red square, stencilValue would be 1. When drawing the green square, stencilValue would be 2. When drawing the yellow square, stencilValue would be 3. Now here's where the problem lies. When drawing the blue square, stencilValue is 4, but I need it to be 1 and then REPLACE the stencil buffer values with 5 where the blue square is drawn. THIS is what I can't figure out how to do. I need a way to draw the blue square ONLY where the stencil buffer is 1, and then continue drawing the light blue square where stencil is 5 and replace with 6. Is this possible using the stencil buffer? I've tried to come up with a scheme using the depth buffer but with no avail. [Edited by - soconne on January 3, 2008 12:48:18 AM]

##### Share on other sites
Well I figured it out and thought I'd share what I found in case anyone else wanted to accomplish the same thing. Assuming each box in the images above are 'containers' you do the following when rendering each one:

drawContainer(w)    render w to framebuffer  for each container c in w     if c is the first container found        glStencilFunc(GL_EQUAL, stencilUsed(w), stencilUsed(w))        glStencilOp(GL_KEEP, GL_KEEP, GL_INC);        stencilUsed(c) = stencilUsed(w) + 1;        lastStencilValue = stencilUsed(c);     else        glStencilFunc(GL_LEQUAL, lastStencilValue + 1, stencilUsed(w));        glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);        stencilUsed(c) = lastStencilValue + 1;        lastStencilValue = stencilUsed(c);     end if  end for loopend function

The above seems to work perfectly fine and does exactly what I needed.

[Edited by - soconne on January 3, 2008 12:57:03 AM]