# Stencil Test - Why use int?

This topic is 3288 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

I'm not 100% sure I understood how it works, but if I did, this is how it works: 1. At the beginning of the frame, the stencil buffer is cleared to an int value (I don't know if negative values are allowed. Are they?) 2. Stencil test. Let's assume each stencil pixel has 32 bits since I need all 32 in my program. The glStencilFunc() mask is applied to ref and to the stencil buffer value, and then this how they are interpreted: the stencil value is interpreted as an unsigned integer, in my case 32-bit, so it's like the C/C++ unsigned int type. ref is clamped to the range [0,2^32-1]. Since ref's maximum value is 2^31-1 (since it's a signed int), clamping is relevant only for negative values: they become zero. And now ref and the stencil value are compared. 3. According to the StencilOp and the glStencilMask(), a value is written or not written into the stencil buffer. If this is true, how am I supposed to use the upper half of the stencil pixel's possible values (from 2^31 to 2^32-1)? I can't set ref to these values because it's an int (why not unsigned int, I don't understand) so if I need to test whether the stencil value is a number above 3^31-1, let's say 4,000,000 , I just can't! I'll have to do a time-consuming trick. For one pixel it's not a problem, but it wastes time for many pixels, few times per frame. I'll need to either read the pixel's value into the RAM and check it there (software test...I want it to be a fast hardware test!) or invert the value, test and invert again. after inverting ref can have the value I need, which is now in int's range. The problem is that I don't want to invert twice for every test I do (and there will be many each frame...) ! Is there another way? Or I am mistaken and negative values can be tested directly? If yes, how?

##### Share on other sites
AFAIK stencil buffers are 8 bit at most (ie, 0-255).

##### Share on other sites
>>Stencil test. Let's assume each stencil pixel has 32 bits since I need all 32 in my program.

before u get carried away did u query the actual stencil bits?
most hardware only has 256 different values

##### Share on other sites
Then I'll have to do more tests per frame and clear the buffer more times... :(

Thanks for the help.

##### Share on other sites
What are you actually trying to do?

##### Share on other sites
According to the GL spec : "Each pixel in a stencil buffer consists of a single unsigned integer value." - §4 Per-Fragment Operations and the Framebuffer
In other words, the stencil buffer is not meant to store negative values.

About the number of values your stencil buffer can store, most graphics card support 8-bit stencil buffer, but if you ask more bits, you may either get a software implementation, your stencil buffer creation may fail, or the driver may give you a 8-bit buffer no matter the number of bits you call to your buffer creation function. In the best case, you can have a deeper stencil buffer with hardware support, but these graphics card are pretty rare, especially gaming cards because games don't need this (note: I'm assuming you're willing to develop a game-oriented 3D engine since you're posting at gamedev.net). Either way, you can call glGetIntegerv with GL_STENCIL_BITS to query how many bits the driver really allocated. Please note that GL_STENCIL_BITS is deprecated in OpenGL 3.0, if you use a forward compatible context, the query will generate a GL_INVALID_OPERATION error.

The important points are :
- if you want your application to run on the majority of graphics card, use a 8-bit stencil buffer, that's the only format you're 99% sure it will be supported correctly,
- if you want your application to run on a specific hardware, simply try to initialize a deeper stencil buffer, check GL_STENCIL_BITS to make sure you got the number of bitplanes you want, and see if the performance is okay or not (if not: it means you fell back to a software pass)

1. 1
2. 2
3. 3
4. 4
frob
15
5. 5

• 16
• 12
• 20
• 12
• 14
• ### Forum Statistics

• Total Topics
632155
• Total Posts
3004477

×