Erik Rufelt

Great work! I really like the feeling.

I always thought this was a nice solution to the issue, albeit more expensive to compute the edges: http://www.sjbrown.co.uk/2004/05/11/continuous-silhouettes/

I've given it zero thought but it seems likely this could be implemented nicely on GPU nowadays.

 

T

That definitely seems like the best solution!

Pretty much did exactly what you said. Definitely fixes the self shadowing issue. However, the shadow shrinking produces a shadow with a jagged, per-polygon edge. So ironically it really just moves the same jagged artifacts off the self-shadows and puts them on the regular shadows. :\

Removing lighting from any edges that touches the shadow instead is an alternative, to grow the shaded part on the caster instead of shrinking the volume, but it's usually a bit more expensive to implement, depending on what rendering techniques you use..

For convex models like the sphere the easiest alternative is of course to not use self-shadowing on the caster, but that won't work on concave models where you want self-shadowing.

Another alternative, since the problem only occurs on the shadow caster, is to fix it by adding the lighting on edge polys on the caster, emulating shrinking the volume only on the caster. Run an extra lighting-pass on the polygons that are at the shadow edge, selected using the same algorithm as when constructing the shadow volumes (geometry-shader?).

So basically draw the model triangles for the caster, as for a lighting-pass, but without stencil test, and with a geometry shader that only outputs the triangles where the triangle-normal faces away from the light, and at least one vertex-normal faces the light. This will discard all polys except those that you know are in the shadow volume but have at least one normal that is facing the light. Should be relatively cheap.

It all depends on what artifacts or deviations from "correct" shadows are acceptable. The problem with smooth normals and exact silhouette shadows is that the silhouette is an approximation of a smooth surface, and the interpolated normals attempts to hide that fact. For perfectly hard shadows, the jagged edge on the sphere caster is actually correct.

Another way that might solve some things is by shrinking the silhouette inwards by moving the vertices toward the center instead of removing vertices, which in some circumstances can probably solve the problem, but of course has the potential to bring new artifacts.

I assume you combine your stencil shadows with drawing lighting with vertex-normals.

The problem occurs when normals are interpolated for smooth shading. If you draw your sphere with flat shading, as in that every triangle has only one normal, and not interpolated vertex normals, then it would look as expected. The reason for this is that stencil-volume edges are extruded between two triangles where one faces the light and one faces away from the light. With interpolated vertex-normals the normals used in the lighting calculation don't match the normals used to calculate the stencil shadow.

If every vertex that touches a triangle that is back-facing with respect to the light receives zero lighting, then the problem goes away.

You can fix it by, when you extrude the edges between back and front-facing polygons, instead of using the triangle plane normal to determine facing, looping over the three vertex normals in the triangle and checking if any of the vertex normals face the light. If at least one normal faces the light, then consider the triangle front-facing.

This will however cause your shadow volumes to shrink a bit, which may or may not be acceptable.

Another variation is to change the vertex-normals so that any vertex that touches a back-facing triangle receives zero lighting, which would keep your shadow volumes the same, but cause the shadow edge to grow a bit instead, as the shadow would be blurred into the lighted part instead of the light being blurred into the shadowed part.

Unfortunately, I can't really make a minimal sample; an identical program's source works in one project, but the same code fails when recompiled in a different project. It's very bizarre.

Run a diff on the project files to find exactly what lines are different, then change one after another until it works.

If you make a minimal example, depending on your environment, that should be just one .cpp file (identical) and one project file (different).