Per-polygon self shadowing problem with stencil shadows

Recommended Posts

mv348    263
I'm facing a somewhat obvious problem with shadow volumes that I somehow failed to anticipate. I just wanted a few tips and suggestions on how to deal with it.

We love shadow-volumes because they provide perfect resolution shadows with none of the aliasing artifacts that are characteristic of shadow maps. But the only case where this fails is on the occluder itself. Since the silhouette edge is calculated on a per-polygon basis, the self shadowing can appear jagged.

I couldn't get a clear screenshot so I just drew up a pic.

[attachment=15525:silhoette-jaggies.jpg]

What are some common ways to deal with this?

So far my ideas are:

1. Blur the shadow a lot.
2. Use very high polygon models with stencil shadows
3. Use gpu tesselation to make polygons finer
4. Use a hybrid system with a shadow-map to handle self shadowing.

1 works a bit but still creates noticable artifacts.

2 is very limiting

3 is maybe a good idea but complicated as it involves not only subdividing the mesh, but also keeping track of adjacent vertices for silhouette edge calculation

4 seems like a good idea except the main reason i'm using stencil shadows is that shadow maps are not greatly suited for point lights.

Ideas and suggestions would be most appreciated!

Share on other sites
Erik Rufelt    5901

I assume you combine your stencil shadows with drawing lighting with vertex-normals.

The problem occurs when normals are interpolated for smooth shading. If you draw your sphere with flat shading, as in that every triangle has only one normal, and not interpolated vertex normals, then it would look as expected. The reason for this is that stencil-volume edges are extruded between two triangles where one faces the light and one faces away from the light. With interpolated vertex-normals the normals used in the lighting calculation don't match the normals used to calculate the stencil shadow.

If every vertex that touches a triangle that is back-facing with respect to the light receives zero lighting, then the problem goes away.

You can fix it by, when you extrude the edges between back and front-facing polygons, instead of using the triangle plane normal to determine facing, looping over the three vertex normals in the triangle and checking if any of the vertex normals face the light. If at least one normal faces the light, then consider the triangle front-facing.

This will however cause your shadow volumes to shrink a bit, which may or may not be acceptable.

Another variation is to change the vertex-normals so that any vertex that touches a back-facing triangle receives zero lighting, which would keep your shadow volumes the same, but cause the shadow edge to grow a bit instead, as the shadow would be blurred into the lighted part instead of the light being blurred into the shadowed part.

Edited by Erik Rufelt

Share on other sites
Kryzon    4629

Hi mv348, I have a very similar opinion to yours regarding regarding the self-shadowing back faces - it's there, and it's a problem that needs to be dealt with.

Fortunately, after some research I got to know the "Reverse-Extruded Shadow Volumes" technique by Renaldas Zioma.

In simple terms, to create the volume, instead of extruding the back faces and leaving the front faces in place, you do the opposite.

This technique is useful for two reasons: It works flawlessly with using a lower-detail mesh to generate your volume (so it's more performant); It also places any self-shadowing back faces exclusively in the ambient-lit region of the object - effectively hiding these hard edges if you use ambient shadows instead of the common "transparent stenciled-quad" shadows.

One way to do ambient shadows is to render your scene with lights and then do another pass (using the stencil) with ambient-only on all shadowed areas.

In any case, you can find literature for this technique here:

http://aras-p.info/texts/revext.html (includes a demo)

http://tog.acm.org/resources/shaderx/ (Shader X²: Tips & Tricks, Section V - Shadows)

With this solved, you're still left with having to avoid the Z-Fail method patented by Creative Labs. Have you had any progress on this?

Share on other sites
mv348    263

Thanks for the detailed responses, Erik Rufelt, and Kryzon. I don't have time to go over your suggestions in detail at this moment, but will do so tomorrow.

With this solved, you're still left with having to avoid the Z-Fail method patented by Creative Labs. Have you had any progress on this?

I'm not sure what you are asking here. Is the depth-fail method for testing shadow-volumes considered a proprietary method of creative labs? O.o that would suck.

Share on other sites
Kryzon    4629

I'm surprised you weren't aware of this.

I can understand it, however. Patenting a graphic technique is perhaps foul play? (Imagine if normal-mapping and others were under the same condition).

Method for Rendering Shadows using a Shadow Volume and a Stencil Buffer

Stencil-shadows pretty much died because of this and from the fact that they don't scale well with higher-detail meshes, which was the natural prospect for game engines.

But with mobile specs and some lower-end consoles (the Wii, for instance) they can present a reasonable method again.

There are ways around camera intersection of shadow volumes that don't rely on depth-fail to render stencil shadows, but they're more complex and in some cases less accurate.

I'm not a lawyer. I was hoping something as simple as changing the stencil operators would not infringe the patent; After all, it wouldn't follow the description of the patent anymore.

Can anyone shed some light (no pun intended) on this? I'm referring to this post here (Shadow Volume SceneProcessor) where the OP claims at the end of his post of not infringing the patent by using different operators for the stencil function.

Edited by Kryzon

Share on other sites
Hodgman    51234

@mv348, do you use a high-poly mesh for rendering, but a low-poly mesh for shadows? Or do you use the same mesh for both?

As long as you're only extruding truly back-facing edges, this artefact shouldn't occur, or more specifically, it should only occur in areas that are un-lit, so you can't notice.

I'm not a lawyer. I was hoping something as simple as changing the stencil operators would not infringe the patent; After all, it wouldn't follow the description of the patent anymore.
Can anyone shed some light (no pun intended) on this? I'm referring to this post here (Shadow Volume SceneProcessor) where the OP claims at the end of his post of not infringing the patent by using different operators for the stencil function.

I am not a lawyer either, but I'd personally go ahead using modified versions of Z-fail stencil shadows like that one, without worrying too much.

If you read the wording on the creative labs patent, it's very specific in it's claims and therefore easy to circumvent.

For one, it explicitly claims rendering the front and back faces separately  in two distinct passes. These days GPUs support two-sided stencil testing, where we can render the front and back faces in a single pass (with different stencil tests for each). This bypasses creative's claims and implements pretty much the same algorithm without using the method that's owned by them.

Share on other sites
Kryzon    4629

Thanks for confirming. This should make it much simpler then.

On the subject of lower-density meshes for generating the volumes, this technique was used on Shadow of the Colossus (PS2). They also blurred the resulting shadows to get soft edges. Their paper has some illustrations of it: Graphics used in Shadow of the Colossus

Share on other sites
mv348    263

Thanks again to you all, for your helpful fee

Share on other sites
mv348    263

Thanks again to you all, for your helpful feedback and suggestions.

@mv348, do you use a high-poly mesh for rendering, but a low-poly mesh for shadows? Or do you use the same mesh for both?

Using the same for both. Its relatively low poly.

So I tried implementing what Erik Rufelt suggested:

You can fix it by, when you extrude the edges between back and front-facing polygons, instead of using the triangle plane normal to determine facing, looping over the three vertex normals in the triangle and checking if any of the vertex normals face the light. If at least one normal faces the light, then consider the triangle front-facing.

This will however cause your shadow volumes to shrink a bit, which may or may not be acceptable.

Pretty much did exactly what you said. Definitely fixes the self shadowing issue. However, the shadow shrinking produces a shadow with a jagged, per-polygon edge. So ironically it really just moves the same jagged artifacts off the self-shadows and puts them on the regular shadows. :\

@Hodgeman, you said:

As long as you're only extruding truly back-facing edges, this artefact shouldn't occur, or more specifically, it should only occur in areas that are un-lit, so you can't notice.

Are you suggesting the same technique as Erik? What do you mean by a truly back-facing edge?

@Kryzon, you said:

Fortunately, after some research I got to know the "Reverse-Extruded Shadow Volumes" technique by Renaldas Zioma.

In simple terms, to create the volume, instead of extruding the back faces and leaving the front faces in place, you do the opposite.

I fail to see how extruding the front-faces instead makes the self-shadowing problem any better. Don't have time to read through and digest a paper right now. Could you give me a few more details on how they resolve the self-shadowing issue? I already have ambient shadows in place. (which I assume means, only ambient light is rendered in shadowed pixels)

Edited by mv348

Share on other sites
Erik Rufelt    5901

Pretty much did exactly what you said. Definitely fixes the self shadowing issue. However, the shadow shrinking produces a shadow with a jagged, per-polygon edge. So ironically it really just moves the same jagged artifacts off the self-shadows and puts them on the regular shadows. :\

Removing lighting from any edges that touches the shadow instead is an alternative, to grow the shaded part on the caster instead of shrinking the volume, but it's usually a bit more expensive to implement, depending on what rendering techniques you use..

For convex models like the sphere the easiest alternative is of course to not use self-shadowing on the caster, but that won't work on concave models where you want self-shadowing.

Another alternative, since the problem only occurs on the shadow caster, is to fix it by adding the lighting on edge polys on the caster, emulating shrinking the volume only on the caster. Run an extra lighting-pass on the polygons that are at the shadow edge, selected using the same algorithm as when constructing the shadow volumes (geometry-shader?).

So basically draw the model triangles for the caster, as for a lighting-pass, but without stencil test, and with a geometry shader that only outputs the triangles where the triangle-normal faces away from the light, and at least one vertex-normal faces the light. This will discard all polys except those that you know are in the shadow volume but have at least one normal that is facing the light. Should be relatively cheap.

It all depends on what artifacts or deviations from "correct" shadows are acceptable. The problem with smooth normals and exact silhouette shadows is that the silhouette is an approximation of a smooth surface, and the interpolated normals attempts to hide that fact. For perfectly hard shadows, the jagged edge on the sphere caster is actually correct.

Another way that might solve some things is by shrinking the silhouette inwards by moving the vertices toward the center instead of removing vertices, which in some circumstances can probably solve the problem, but of course has the potential to bring new artifacts.

Edited by Erik Rufelt

Share on other sites
Hodgman    51234
Can you post an actual screenshot of your artefact? Does the light contribution from the shadow-casting light reach zero before the edge of the shadow volume? Does the shadow volume only remove the light contribution from this one light?

Share on other sites
mv348    263

Screenshots are hard to get of this since there's so much flicker and the artifacts aren't too noticeable unless the flicker is there. So I took a video clip:

For this video I reverted back to using only the triangle plane normals to calculate front or back facing. I found that offsetting the shadow volume polygons by epsilon*lightDirection (this is currently only directional light shadows) works well in eliminating a lot of the artifacts, but as you can see on the torus, its not quite gone. Even if I optimize epsilon for just the torus object I can't get it to look right.

I like your suggestions, Erik Rufelt. I really am tempted to try adding an extra pass to deal with this directly, but I'm worried that it will cost double the processing time per shadow.

Have a look at the video clip and see if you come up with some suggestions.

Edited by mv348

Share on other sites
Erik Rufelt    5901

I always thought this was a nice solution to the issue, albeit more expensive to compute the edges: http://www.sjbrown.co.uk/2004/05/11/continuous-silhouettes/

I've given it zero thought but it seems likely this could be implemented nicely on GPU nowadays.

T

That definitely seems like the best solution!

Share on other sites
mv348    263

Hello friends. I liked the link posted by Tesselator I set about to implementing it and I've gotten quite close I think. I've tested that it correctly emits the silhouette edges exactly where the non-ambient lighting reaches 0.

The only problem that remains is shown below. (Never mind the diamond shaped region at the left, that's a model in the background with non-continuous surface normals, which I need to add special case code to handle)

The lighting shader is currently set to draw green where the shadow will be. The Z-fighting you see on the upper half is not really an issue because the shadow only blocks non-ambient light, and so it will have no affect on the lighting there.

The real problem is the extreme right and left interior faces of the torus. The light is pointed in the negative Y direction. The shadow volume is projected straight down, and these extreme edges do not quite catch the shadow.

I have tried a few tricks to enlarge the projected shadow. For example, If I was projecting a shadow volume from triangle (v0,v2,v4), I would instead project (v0-eps*n0, v2- eps*n2,v4-eps*n4) where n0,n2,n4 are the respective normals for v0,v2,v4 and epsilon is a small positive value. This actually worked very well except it created gaps in the shadow volume, producing narrow (or in some cases fairly wide) line artifacts.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account