First off, I consider myself pretty adept with both OpenGL as well as WinAPI, so I've hopefully managed to exclude any dead obvious explanations (then again, you'll never know...). I'm trying to convert a GLUT-based sample application to a straightforward WinAPI-based application. I've never really taken an interest in GLUT and it's always seemed as something overly straightforward to me. However, in this case something (probably really simple) is happening that is genuinely puzzling me. I've narrowed the problem down to what is most likely the window creation code. The sample application uses the stencil buffer to segment a projective texture, so the bulk of the attention should rightfully be on the stencil buffer. However, GLUT requires an additional alpha bit (GLUT_ALPHA) to be specified for the window to work in RGBA mode. Specs say: "Bit mask to request a window with an alpha component to the color buffer(s)." This makes little sense to me since the window is (in Windows) initialized through CreateWindow() and the pixel format descriptor should already include the RGBA mode, which should ensure the presence of an alpha channel. Just to be sure I wasn't misunderstanding, I tried setting the alpha buffer in the pixel format descriptor to 1, which only seemed to make matters worse (no projection drawn at all). The effect of the demo (sadly I am not in a position to post screenshots) is like this: I The original demo using GLUT draws the projected texture properly with appropriate bits cut out using a simple stencil test. The initialization code looks like this:

   glutInit(&argc, argv);
   glutInitDisplayMode(GLUT_DOUBLE|GLUT_DEPTH|GLUT_RGB|GLUT_ALPHA|GLUT_STENCIL);

II My adoption of that code draws the projected texture properly, but does not render the occluded areas properly (no occluding occurs through the stencil buffer) at all. That is, the projection blasts through everything as if no stencil tests were ever performed. The pixel format initialization code looks like this:

   PIXELFORMATDESCRIPTOR pfd= {
      sizeof(PIXELFORMATDESCRIPTOR), 1,
      PFD_DRAW_TO_WINDOW | PFD_SUPPORT_OPENGL |PFD_DOUBLEBUFFER,
      PFD_TYPE_RGBA,    //4-component mode
      24,		//color depth
      0, 0, 0, 0, 0, 0,
      0,                //no alpha buffer
      0, 0, 0, 0, 0, 0,
      24,               //depth buffer
      1,                //stencil buffer
      0,PFD_MAIN_PLANE, 0, 0, 0, 0 };

I'm fairly certain the problem is not in the stencil buffer, however, because I reverted to trying the code out without any alterations at all other than switching the GLUT windowing code to my own. I could probably explain the problem more closely, although without the ability to post screenshots, I'm feeling a little helpless without any suggestive questions. As I mentioned, the problem is most likely in the GLUT_ALPHA flag - can someone at least explain what that does, because even Google doesn't seem to know what the heck it really is. Thank you for your time [Edited by - irreversible on November 12, 2005 7:16:05 AM]

Calling glGetIntegerv() for GL_STENCIL_BITS returns 8 regardless of the number of bits defined in the pixel format descriptor, so I suppose the buffer is always initialized to 8 bits (at least on my GF2).

What's more iteresting, is that calling glGetIntegerv() for GL_ALPHA_BITS in the GLUT implementation returns 8. However, setting the alpha buffer 8 bits (similarly to the stencil buffer) in the WinAPI version causes no projected texture to be drawn at all. There are also some other discrepancies if the alpha buffer is enabled, but those fall beyond the scope of my question in this case.

The behaviors simply do not match, so I guess there's something more that I'm missing...

LilBudyWizer - yes. I did not alter the render code, though. At least not consciously (that is, unless GLUT does something behind the scenes, everything should be the same).

zedzeek - I don't need more than 8 bits. I just wanted to express that setting either the alpha or stencil bits to 1 (or any other number) in the PFD automatically uses 8 bits for the respective buffer, that's all.

As I mentioned, if I don't initialize the alpha buffer, the projection is drawn, but not clipped. If I do initialize it, no projection at all is drawn... That's what's confusing me.

Incidentally - I was browsing around in docs on PIXELFORMATDESCRIPTOR and - I don't know how I missed this before, but:

cAlphaBits
Specifies the number of alpha bitplanes in each RGBA color buffer. Alpha bitplanes are not supported.
cAlphaShift
Specifies the shift count for alpha bitplanes in each RGBA color buffer. Alpha bitplanes are not supported.

This is really really confusing because GLUT_ALPHA says:

GLUT_ALPHA
Bit mask to request a window with an alpha component to the color buffer(s).



It is stated that GLUT will not initialize the alpha channel properly unless GLUT_ALPHA is specified, even though GLUT_RGBA may be specified (looking up GLUT_RGBA in glut.h, we'll find that it's defined as:

#define GLUT_RGBA GLUT_RGB)

This makes no sense, simply because PIXELFORMATDESCRIPTOR doesn't seem to provide a means to initialize the color buffer without an alpha component anyway (the only eligible format choice for OpenGL is PFD_TYPE_RGBA, which should already include the four channels). Does GLUT somehow implement an additional alpha channel internally if GLUT_ALPHA is specified? If so then how exactly.


Furthermore - calling:

glutInitDisplayMode(GLUT_DOUBLE|GLUT_DEPTH|GLUT_RGBA|GLUT_ALPHA|GLUT_STENCIL);
glGetIntegerv(GL_STENCIL_BITS, &stencil);
glGetIntegerv(GL_ALPHA_BITS, & alpha);

returns stecil and alpha both as 8 bits.

Failing to specify the alpha bitplane in the PIXELFORMATDESCRIPTOR in a Windows-specific example will return alpha as 0 (however, it will be 8 if the alpha bitplane is specified). Again, this makes little sense if you think about it for a moment.

Has no one really had a problem similar to this?