# Buggy Shadow Volume / zfail

## Recommended Posts

Ive been stuck on shadow volumes now for a while. almost there, always a last bug. I have a 'tree' mesh(es), which is a bunch of cones (made of triangles) stuck together like branches. Each cone is a solid model, each triangle wound the same (glcullface clips all in or all out) Im sure that this is well for the zfail, cause one cone produces a proper shadow (jsut like a simple cube goes well) but when the meshes overlap it forms a problem. At least, my guess is that seperate volumes cannot overlap or something (like when i 'stick' one cone in the other to make a 'branch'). however, when i have two cubes, and drag one in the other by hand, all seems well... the bug is that, only in the surfaces of the shadow, there are lines. also, the front and back cap; - the front cap is all triangles that are NOT facing the light ('back' of the mesh) - the back cap is the front-cap extruded to inf - when normally rendering the volume with culling enabled, you can see every side, when outside (and see nothing when inside) right ???? and a stupid question, should the back polygons be affected / be shaded? or only cast a shadow ? (of course it should be shaded, but does the zfail do that? i know it selfshadows, but the back faces seem different somehow) thanx

##### Share on other sites

If you are using a modeling program then just weld the verts from the tree's branch to the tree trunk, tidying up the seam.
If you want to stick to a cone, then ensure the base of the cone is closed. This will of course require extra triangles in your volume, but you only have to render them to the stencil buffer (not color/depth as they won't show unless you're inside the tree! - and that's regardless of viewing the extra triangles from front/back).

Hence, your second question...you also do not need to draw *any* back faces to color/depth buffers, only the stencil buffer like you are.

##### Share on other sites
my cones have a base. they are solid and wounded. so i prety much rules out the geometry of one cone since they also produce a good shadow. its when they overlap. But understanding you, that shouldn't be a problem right ??

##### Share on other sites
Apologies for not understanding the first time, you did clearly state the cones were solid.

The lines you describe, does every triangle of the cone produce them, or just the silhouette, or perhaps just the odd one here and there? And these lines are un-shadowed areas?

Another question, you say that two overlapping box's shadows never have this bug?

##### Share on other sites
Quote:
 each triangle wound the same (glcullface clips all in or all out)

Not sure what you mean there (all in or all out)? Your volume's triangles should be defined the same way as you would if you were building a standard model.

##### Share on other sites
right, thats what i mean. The inside is culled (standard), so you would see the volume as normal. completely visible when outside, completely invisible when inside.

##### Share on other sites
There is no reason why an object can not penetrate another. I have various meshes overlapping (car / driver) working perfectly.

I guess a screenshot of the bug will really help here.
(http://www.tinypic.com/ are a good and fast host)

edit: I've just remembered a similar sounding bug which I used to have.
when drawing *front* faces of your volume's caps (not walls)
glEnable( GL_POLYGON_OFFSET_FILL );glPolygonOffset( 1.0f, 1.0f );...draw...glDisable( GL_POLYGON_OFFSET_FILL );

Without using the above I get "lines" too.

[Edited by - jezham on October 13, 2008 3:23:14 PM]

##### Share on other sites
have also included that, but still those lines...

##### Share on other sites
Perhaps try negative values in your situation (I draw lit areas first)? It really is tricky to say without seeing an image.

##### Share on other sites
i tried al kinds of values.
at least its good to know that overlapping should be ok.
can you post me your pass code, or take a look at mine

glEnable (GL_POLYGON_OFFSET_FILL);
glPolygonOffset (1.0f, 1.0f);

// disable gl effects
glDisable (GL_LIGHTING);
glDisable (GL_BLEND);

// disable color and depth masks
glDepthFunc (GL_LEQUAL);

// enable culling
glEnable (GL_CULL_FACE);
// setup stencil buffer
glEnable (GL_STENCIL_TEST);

// first pass
glCullFace (GL_FRONT); // Set Culling Face To Back Face
glStencilFunc (GL_ALWAYS, 0x0, 0xff);
glStencilOp (GL_KEEP, GL_INCR, GL_KEEP);
rdrsVolume (100.0f);

// Second Pass. Decrease Stencil Value In The Shadow
glCullFace (GL_BACK); // Set Culling Face To Back Face
glStencilFunc (GL_ALWAYS, 0x0, 0xff);
glStencilOp (GL_KEEP, GL_DECR, GL_KEEP);
rdrsVolume (100.0f);

glDisable (GL_POLYGON_OFFSET_FILL);

// set front face

glDisable (GL_CULL_FACE);

// Enable Rendering To Colour Buffer For All Components

// setup stencil
glStencilFunc(GL_NOTEQUAL, 0x0, 0xff);
glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE);

Vector clr = {0,0,0, 0.9f};
// render overlay
mod.renderFilter (clr);

thanx

##### Share on other sites
to answer you question (missed them, sorry)

"The lines you describe, does every triangle of the cone produce them, or just the silhouette, or perhaps just the odd one here and there? And these lines are un-shadowed areas?"

picture one floor quad (for the shadow), 1 tall cone (the 'tree trunk'), with a bunch of other cones sticking out of that one ('tree branches'). The light is above it at the right, so it casts the shadow to the left.
The lines seem to originate from the overlapped areas, where 'branches spawn' (or the bases of the cones but they seem right..) the lines run NE-SW diagonally, displayed only in the surface of shadow (so the floor quad and tree itself). And the negate the shadow (so cancel shadow where there is, and vice versa) but are straight lines to infinity.....

"Another question, you say that two overlapping box's shadows never have this bug?"
i can place simple cubes and drag them around. their shadows went well, also when i dragged them inside eachother (but should verify this again to be real sure)

thanx

##### Share on other sites
You should disable polygon offset before pass 2.

I'm not sure that will solve your problem though. Have a look through my code to see if anything lights up
		glCullFace( GL_BACK );								//draw only the front faces of the shadow volume		//clear the stencil buffer		glClear( GL_STENCIL_BUFFER_BIT );		//store current OpenGL state		glPushAttrib( GL_DEPTH_BUFFER_BIT | GL_LIGHTING_BIT | GL_STENCIL_BUFFER_BIT );			//pass 1 (scene)			//draw scene with light(s) enabled			Draw...( true );			...			//pass 2 (volumes)			glDisable( GL_TEXTURE_2D );			//store current OpenGL state			glPushAttrib( GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_POLYGON_BIT | GL_STENCIL_BUFFER_BIT );				glColorMask( 0, 0, 0, 0 );					//do not write to the color buffer				glDepthMask( 0 );							//do not write to the depth (Z) buffer				glEnable( GL_STENCIL_TEST );				//enable stencil testing				glStencilFunc( GL_ALWAYS, 0, ~0 );			//set the reference stencil value to 0				//				glStencilOp( GL_KEEP, GL_INCR, GL_KEEP );	//increment the stencil value on Z fail				glCullFace( GL_FRONT );						//draw only the back faces of the shadow volume				//draw walls (+ caps)				ShadowVolumes->Draw( shadowVolumes_shader, true );				//				glStencilOp( GL_KEEP, GL_DECR, GL_KEEP );	//decrement the stencil value on Z fail				glCullFace( GL_BACK );						//draw only the front faces of the shadow volume				//draw walls (+ caps)				ShadowVolumes->Draw( shadowVolumes_shader, false );		//call VBO, shader extrudes correct quadrilaterals (plus animates ms3d:)			//restore OpenGL state			glPopAttrib();			//			//pass 3 (scene)			//re-draw scene with light(s) disabled,			glDepthFunc( GL_EQUAL );						//only where it has previously been drawn			glDepthMask( 0 );								//do not write to the depth (Z) buffer			//update the color only where the stencil value is NOT 0			glEnable( GL_STENCIL_TEST );			glStencilFunc( GL_NOTEQUAL, 0, ~0 );			glStencilOp( GL_KEEP, GL_KEEP, GL_KEEP );			//			Draw...( false );			...			}		//restore OpenGL state		glPopAttrib();	}

edit: updated remark in pass 3, where color update happens only when value is NOT 0

[Edited by - jezham on October 13, 2008 9:04:17 PM]

##### Share on other sites
I thought i had it when i saw the line
glStencilOp( GL_KEEP, GL_KEEP, GL_KEEP );

where i use GL_REPLACE,GL_REPLACE,GL_REPLACE.
though, these two seem to produce the same results.
is that right ?? why ?

thanx

##### Share on other sites
iirc it was when I decided that drawing lit areas first would require less fill-rate, so when the un-shadowed areas were skipped I would keep what was there rather than replace with light over shadow.

If I try to replace all, then future volumes cancel out previous ones.

##### Share on other sites
still lines,
Ill recheck my geometry again I guess
thanx though

##### Share on other sites
2 things

1) ensure all normals (per triangle) are the same face normal (not vertex)
2) you might like this one ;) a *single* axis-aligned-billboarded triangle, extruded to form a cones shadow volume (one of my better inventions: I name it the jezume! - also works similarly with boxes, spheres, capsules, and cylinders, and even oblate spheroids)

##### Share on other sites
Im re-inspecting my normals now for the billionth time .... :)
I dont quite get what you mean by 2. but its sounds interesting.
Can you elaborate some maybe?
thnax

(You mean using only one triangle as volume for a cone, instead of edges et...??
that wont suffice for my goals i think)

##### Share on other sites
Keep inspecting your code, as my optimal solution only works with basic primitives (sphere, capsule...). You will still need to define / calculate a silhouette for complex models (shaders are awesome at that), and even boxes.

Ok to elaborate on my 'single' extruded triangle, for your cone. I haven't knocked one up to show you (I will soon), but I've picked a scene which demonstrates it on cylinders and spheres:

Pic 1, the desired effect - self shadowing
Pic 2, overlayed with the shadow volumes

Pic 3, a close up:
Note how the cylinder has 1 'box' for the volume - essentially a plane axis-alligned to face the light, and extruded away to whatever distance.
Note also the sphere's volume - just a disc, billboarded this time to face the light, again extruded away.

Now, for a cone you do the same as with a cylinder, except with a single triangle!
If you view your cone down it's Y-axis then you will have to apply a disc-volume to the cone's base...but as your tree's-trunk's-shadow will overlap then there's no need for the additional disc.

HTH, happy to answer all questions on this - perhaps I'll knock up a tutorial in the future :)

btw: can you spot the bug in pic 3 !?

[Edited by - jezham on October 18, 2008 11:08:51 AM]

##### Share on other sites
Quote:
 Original post by jezhamKeep inspecting your code, as my optimal solution only works with basic primitives (sphere, capsule...)....

I think this trick should work on all convex 3D model instead of just basic primitives.

##### Share on other sites
Quote:
 Original post by ma_htyI think this trick should work on all convex 3D model instead of just basic primitives.

Sure...if they're symmetrical convex* (like the 'egg' shape mentioned above for example), then it will indeed. But the shape needs to be round in nature, to allow rotation around 1+ axis. A box is convex but we know that isn't practical...saying that, a box volume can be drawn at the exact same speed as if my method was possible (with it). Even the cone method is only optimal without the base-disc, else a shader will be just as quick with edge-based-quadrilaterals.

* edit, needless to say it doesn't matter if convex or concave...only matters that they're symmetrical around at least 1 axis (example; a non translucent bottle).

[Edited by - jezham on October 19, 2008 5:51:49 AM]

## Create an account

Register a new account

• ### Forum Statistics

• Total Topics
628403
• Total Posts
2982480

• 9
• 10
• 9
• 21
• 24