Sign in to follow this  
Tom Backton

Stencil Test - Why use int?

Recommended Posts

I'm not 100% sure I understood how it works, but if I did, this is how it works: 1. At the beginning of the frame, the stencil buffer is cleared to an int value (I don't know if negative values are allowed. Are they?) 2. Stencil test. Let's assume each stencil pixel has 32 bits since I need all 32 in my program. The glStencilFunc() mask is applied to ref and to the stencil buffer value, and then this how they are interpreted: the stencil value is interpreted as an unsigned integer, in my case 32-bit, so it's like the C/C++ unsigned int type. ref is clamped to the range [0,2^32-1]. Since ref's maximum value is 2^31-1 (since it's a signed int), clamping is relevant only for negative values: they become zero. And now ref and the stencil value are compared. 3. According to the StencilOp and the glStencilMask(), a value is written or not written into the stencil buffer. If this is true, how am I supposed to use the upper half of the stencil pixel's possible values (from 2^31 to 2^32-1)? I can't set ref to these values because it's an int (why not unsigned int, I don't understand) so if I need to test whether the stencil value is a number above 3^31-1, let's say 4,000,000 , I just can't! I'll have to do a time-consuming trick. For one pixel it's not a problem, but it wastes time for many pixels, few times per frame. I'll need to either read the pixel's value into the RAM and check it there (software test...I want it to be a fast hardware test!) or invert the value, test and invert again. after inverting ref can have the value I need, which is now in int's range. The problem is that I don't want to invert twice for every test I do (and there will be many each frame...) ! Is there another way? Or I am mistaken and negative values can be tested directly? If yes, how?

Share this post


Link to post
Share on other sites
>>Stencil test. Let's assume each stencil pixel has 32 bits since I need all 32 in my program.

before u get carried away did u query the actual stencil bits?
most hardware only has 256 different values

Share this post


Link to post
Share on other sites
According to the GL spec : "Each pixel in a stencil buffer consists of a single unsigned integer value." - §4 Per-Fragment Operations and the Framebuffer
In other words, the stencil buffer is not meant to store negative values.

About the number of values your stencil buffer can store, most graphics card support 8-bit stencil buffer, but if you ask more bits, you may either get a software implementation, your stencil buffer creation may fail, or the driver may give you a 8-bit buffer no matter the number of bits you call to your buffer creation function. In the best case, you can have a deeper stencil buffer with hardware support, but these graphics card are pretty rare, especially gaming cards because games don't need this (note: I'm assuming you're willing to develop a game-oriented 3D engine since you're posting at gamedev.net). Either way, you can call glGetIntegerv with GL_STENCIL_BITS to query how many bits the driver really allocated. Please note that GL_STENCIL_BITS is deprecated in OpenGL 3.0, if you use a forward compatible context, the query will generate a GL_INVALID_OPERATION error.

The important points are :
- if you want your application to run on the majority of graphics card, use a 8-bit stencil buffer, that's the only format you're 99% sure it will be supported correctly,
- if you want your application to run on a specific hardware, simply try to initialize a deeper stencil buffer, check GL_STENCIL_BITS to make sure you got the number of bitplanes you want, and see if the performance is okay or not (if not: it means you fell back to a software pass)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this