# OpenGL How to draw visible shadow volumes, but through transparent polygons too

This topic is 1564 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

Disclaimer: this is older OpenGL, and while I would love to switch to all shaders (not sure if that would be appropriate here even), I'd rather complete what I have instead of switching the whole engine over at this point. It is an older project, but I'd still like to see it wrapped up in its current form.

My project is a 3d platformer, you have cubes jumping around on other cubes and they use shadow volumes to show where they are in relation to the ground. Recently I added crumbling blocks, the kind that disappear a few seconds after you touch them. That's great! I have them set so their alpha is reduced based on how long it's been since you've touched them so they fade away after a few seconds. The problem is: even at alpha zero, they're affecting the depth test for my stencil buffer.

The white/grey blocks are the crumbling ones:

(Don't mind the missing face, it's part of an efficiency measure to remove unused faces that doesn't consider disappearing blocks yet)

The problem you see here is that the shadows, while drawn fine on the left and right, are not appearing in the middle behind the player. That's because there are crumbling blocks there, though they have 0% opacity/alpha. You can even see the outline of the cube just above and to the right of the player because that's the shape cut out of the shadows.

Also: the player automatically gets a silhouette when behind another object, same stencil style test as the shadows, so that's why the player is mostly yellow and shadowed yellow but then turns green at the top. It's explained here.

I don't want to leave anything out I might be missing, so here's the whole code chunk (you can see my comments trying to remember what I wrote... I've gotten better at this since 2012... ):

// Give shadows to everything!
// Blend function not causing any differences if enabled/disabled?
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);

// GL_BLEND is the difference between black shadows and dark versions of the colors underneath
glEnable( GL_BLEND );
// Not sure GL_ALPHA is doing anything, nor ALPHA_TEST
glEnable( GL_ALPHA );
glEnable( GL_ALPHA_TEST );

// Color Mask with everything disabled means it doesn't use this data to draw real shapes/objects
// instead, we're going to use it for a stencil test later

// GL_STENCIL_TEST limits the shadows to being drawn in the stencils, which are the levelShadows
glEnable(GL_STENCIL_TEST);

// Just draw the level shadows as regular polygons, but with the stencil option

glEnableClientState(GL_COLOR_ARRAY);
glEnableClientState(GL_VERTEX_ARRAY);

// specify pointer to vertex array

glPushMatrix();
//glScalef(0.99,100.0,0.99);
//glTranslatef(0.0,-50.505,0.0);

glCullFace(GL_FRONT);
glStencilFunc(GL_ALWAYS, 0x0, 0xff);
// if depth fails (can't see only due to z) then increment pixel's value
glStencilOp(GL_KEEP, GL_INCR, GL_KEEP);

glCullFace(GL_BACK);
glStencilFunc(GL_ALWAYS, 0x0, 0xff);
// if depth also fails for back, decrement it so it stays at zero
// but if one increments and the other does not...
// (we'll come back to that later when we see where to place shadows)
glStencilOp(GL_KEEP, GL_DECR, GL_KEEP);

glPopMatrix();

// deactivate vertex arrays after drawing
glDisableClientState(GL_VERTEX_ARRAY);
glDisableClientState(GL_COLOR_ARRAY);

}

/*
for (int i=0; i<cubeNum; i++) {
// cubeWithinPlayerRange demonstrates noticeable improvements,
// castle level with 4player, sfx and music went from 7fps to 12.
// now goes from 24 to 35fps! Wow!
if (cubeWithinPlayerRange(i,player)) {
}
}*/

for (int i=0; i<cubiorNum; i++) { drawPlayerShadow(i); }

// Where the front and the back are not the same (so front face could be seen but back was obscured)
// that is where a shadow is on the ground
// or that is what I believe NOTEQUAL and REPLACE are being used for
// all three replaces means no matter what (fail/zfail/succeed), we will the pixel with the new color
glStencilFunc(GL_NOTEQUAL, 0x0, 0xff);
glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE);
glDisable(GL_STENCIL_TEST);

glDisable( GL_BLEND );
}

// 100% copied from draw_shadow for testing purposes
// full permission to copy this though, according to Josh Beam:
// Fills the stencilled area with a shadow of 50% opacity on top of everything else
glPushMatrix();
glMatrixMode(GL_PROJECTION);
glPushMatrix();
glOrtho(0, 1, 1, 0, 0, 1);
glDisable(GL_DEPTH_TEST);

glColor4f(0.0f, 0.0f, 0.0f, 0.5f);
glVertex2i(0, 0);
glVertex2i(0, 1);
glVertex2i(1, 1);
glVertex2i(1, 0);
glEnd();

glEnable(GL_DEPTH_TEST);
glPopMatrix();
glMatrixMode(GL_MODELVIEW);
glPopMatrix();
}


The function drawItemShadows is the exact one for the crumbling block shadows, but it works the same way as what you see here already. It increments the stencil's pixel value for front faces and decrements it for back faces, so it should be zero if both faces are visible or both are obscured, but if only one is then that's where a shadow lands. But again, the problem is that the z test there isn't taking into account transparent polygons. Before this code is where I draw the items and world itself (shadows are drawn last) and those are drawn with 3 point polygon indexes and 4 point color indexes, 4th slot being alpha.

So how do I skip alpha in my depth tests, but then apply that alpha to the shadow I draw ultimately? Is that possible, or do I need to break this up into separate passes somehow?

##### Share on other sites

Any chance you can sidestep the problem by having your disintegrating blocks flash on and off before disappearing instead of fading out?

Or maybe they can be broken up into lots of tiny voxels that disappear one at a time, or they could scale down to nothing?

Basically, shadow mapping with semi-transparent objects is a hard problem in general. I suppose you might have some luck drawing the shadows of the semi transparent objects separately, but it might be tricky to get them to appear on the player.

##### Share on other sites

The problem is: even at alpha zero, they're affecting the depth test for my stencil buffer.

Transparent objects shouldn't render depth. Won't the problem go away if you disable depth writes for the crumbling blocks?

##### Share on other sites

The problem is: even at alpha zero, they're affecting the depth test for my stencil buffer.

Transparent objects shouldn't render depth. Won't the problem go away if you disable depth writes for the crumbling blocks?

This is a lot better, but doesn't solve everything:

So, no depth test on crumbling blocks = awesome shadows passing through!

Heck, they go so far through it's like the crumbling blocks aren't even there!

...ah shoot, it's like the crumbling blocks aren't even there. Now there are no shadows on top. As you can see, I've added some crumbling blocks above the other crumbling blocks. With the old way, the shadow recognizes the blocks and lands on it.

But the new way makes that shadow invisible.

So what can I do to address that? Multiple passes won't work because if it goes...

noDepth crumblingBlocks

depth crumblingBlocks

...well, for one: it does this funky shadow thing:

That's because the shadow is applied twice to everything at the bottom (I could probably clean that up by finding the difference of the two stencils though and not drawing anything twice) but two:

If I look through transparent crumbling blocks at the shadow of other blocks on TOP of crumbling blocks... nothing appears. I would have to do one of these passes for every single vertical layer of my game world. :|

So more dilemma! I want both to use the blocks with depth test, so I can draw shadows on top of them, but also without so I can draw shadows beneath them. Ah... any tips on this?

##### Share on other sites

So more dilemma! I want both to use the blocks with depth test, so I can draw shadows on top of them, but also without so I can draw shadows beneath them. Ah... any tips on this?

Yeah, that isn't going to work. You can't have transparent surfaces both casting and receiving shadows with less than one pass per layer.

##### Share on other sites

Alright, final product is a combination of the partial solution and a workaround. When you touch a block, it immediately switches to 50% opacity and disables itself as a shadow surface, but also doesn't render before shadows again until after it's back at 100%. The shadows disappear when the crumbling blocks are touched then, but appear as soon as they're whole again, and all transparencies look good.

So it's about 80% of what I hoped for, and it looks good. :) Thanks guys!

1. 1
Rutin
26
2. 2
3. 3
4. 4
5. 5

• 9
• 11
• 10
• 13
• 20
• ### Forum Statistics

• Total Topics
632948
• Total Posts
3009389
• ### Who's Online (See full list)

There are no registered users currently online

×