Shadow Volume Generation Precision
So I have reimplemented my old shadow volumes algorithm--mainly for completeness. This implementation uses a geometry shader to generate depth-fail (read: two-ends capped) geometry.
All works well, except that for some glancing angles, the geometry is extruded out from where it shouldn't. This results in artifacts. This seems to happen particularly where a triangle's edge that should be shadowed actually thinks it's a silhouette edge.
Here's a simple example. All seems well:
But rotate it, and an artifact develops:
Here's a closeup of such an artifact:
The dark face at the bottom should be completely shadowed, and therefore should not generate an edge along the bottom-left. It generates one anyway, screwing up the render.
The normal used to compare to the light direction is generated from the cross product of the geometry itself, so it's not an interpolation issue. Also, the artifacts seem to only happen in a narrow range of glancing angles. The same face will correct itself if turned further.
My question is, is this a common thing? The only thing I could think of is a precision issue, but the error is really quite large. I realize I haven't given much information about the implementation, but does anyone seen this sort of thing with volume extrusion before?
-G
How are you performing your silhouette detection? It looks as though you are just translating the back-faces of the geometry away from the light source, which seems simple enough but might be causing the complication which manifests as a graphical artifact.
In the geometry shader (input GL_TRIANGLES_ADJACENCY):
--A vector L from the triangle's first vertex to the light is computed.
--The normal N is computed by a cross product of the vertices.
--L is then dotted with N
--The sign of the dot product is checked. This determines the side facing the light.
This formula is computed for each of the four triangles. If any of the three adjoining edges jumps from a facing to a non-facing value, then the edge is extruded. To avoid double extrusion, the extrusion only happens if the center triangle is facing the light.
Update: I have modified the geometry shader to show the output more carefully. It seems that the computation of whether an adjacent triangle faces the light is incorrect in some cases.