Sign in to follow this  
soconne

**SOLVED** Help with applying stencil buffer to a problem

Recommended Posts

I'm currently attempting to use the stencil buffer to correctly clip nested rectangles when drawn to the screen. Here is a screenshot of what I want to occur. So the order is this:
  1. Clear stencil buffer to 1
  2. Draw red square where stencil buffer ONLY equals 1, and replace with 2
  3. Draw green square where stencil buffer ONLY equals 2, and replace with 3
  4. Draw yellow square where stencil buffer ONLY equals 3, and replace with 4
  5. Draw blue square where stencil buffer ONLY equals 1, and replace with 5
  6. Draw light blue square where stencil buffer ONLY equals 5, and replace with 6
The problems I'm having come into play in step 6. The blue square isn't shown. The way I'm setting the stencil state values is as so: glStencilFunc(GL_EQUAL, stencilValue, stencilValue); glStencilOp(GL_KEEP, GL_KEEP, GL_INCR); stencilValue is the value in the stencil buffer where drawing is allowed. So when drawing the red square, stencilValue would be 1. When drawing the green square, stencilValue would be 2. When drawing the yellow square, stencilValue would be 3. Now here's where the problem lies. When drawing the blue square, stencilValue is 4, but I need it to be 1 and then REPLACE the stencil buffer values with 5 where the blue square is drawn. THIS is what I can't figure out how to do. I need a way to draw the blue square ONLY where the stencil buffer is 1, and then continue drawing the light blue square where stencil is 5 and replace with 6. Is this possible using the stencil buffer? I've tried to come up with a scheme using the depth buffer but with no avail. [Edited by - soconne on January 3, 2008 12:48:18 AM]

Share this post


Link to post
Share on other sites
Well I figured it out and thought I'd share what I found in case anyone else wanted to accomplish the same thing. Assuming each box in the images above are 'containers' you do the following when rendering each one:


drawContainer(w)

render w to framebuffer

for each container c in w
if c is the first container found
glStencilFunc(GL_EQUAL, stencilUsed(w), stencilUsed(w))
glStencilOp(GL_KEEP, GL_KEEP, GL_INC);
stencilUsed(c) = stencilUsed(w) + 1;
lastStencilValue = stencilUsed(c);
else
glStencilFunc(GL_LEQUAL, lastStencilValue + 1, stencilUsed(w));
glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);
stencilUsed(c) = lastStencilValue + 1;
lastStencilValue = stencilUsed(c);
end if
end for loop
end function



The above seems to work perfectly fine and does exactly what I needed.

[Edited by - soconne on January 3, 2008 12:57:03 AM]

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this