• Create Account

Interested in a FREE copy of HTML5 game maker Construct 2?

We'll be giving away three Personal Edition licences in next Tuesday's GDNet Direct email newsletter!

#ActualErik Rufelt

Posted 04 May 2013 - 08:33 AM

I assume you combine your stencil shadows with drawing lighting with vertex-normals.

The problem occurs when normals are interpolated for smooth shading. If you draw your sphere with flat shading, as in that every triangle has only one normal, and not interpolated vertex normals, then it would look as expected. The reason for this is that stencil-volume edges are extruded between two triangles where one faces the light and one faces away from the light. With interpolated vertex-normals the normals used in the lighting calculation don't match the normals used to calculate the stencil shadow.

If every vertex that touches a triangle that is back-facing with respect to the light receives zero lighting, then the problem goes away.

You can fix it by, when you extrude the edges between back and front-facing polygons, instead of using the triangle plane normal to determine facing, looping over the three vertex normals in the triangle and checking if any of the vertex normals face the light. If at least one normal faces the light, then consider the triangle front-facing.

This will however cause your shadow volumes to shrink a bit, which may or may not be acceptable.

Another variation is to change the vertex-normals so that any vertex that touches a back-facing triangle receives zero lighting, which would keep your shadow volumes the same, but cause the shadow edge to grow a bit instead, as the shadow would be blurred into the lighted part instead of the light being blurred into the shadowed part.

#1Erik Rufelt

Posted 04 May 2013 - 08:29 AM

I assume you combine your stencil shadows with drawing lighting with vertex-normals.

The problem occurs when normals are interpolated for smooth shading. If you draw your sphere with flat shading, as in that every triangle has only one normal, and not interpolated vertex normals, then it would look as expected. The reason for this is that stencil-volume edges are extruded between triangles where one face the light and one faces away from the light. With interpolated vertex-normals the normals used in the lighting calculation doesn't match the normals used to calculate the stencil shadow.

If every vertex that touches a triangle that is back-facing with respect to the light receives zero lighting, then the problem goes away.

You can fix it by, when you extrude the edges between back and front-facing polygons, instead of choosing the triangle plane normal to determine facing, loop over the three vertex normals in the triangle and check if any of the vertex normals face the light. If at least one faces the light, then consider the triangle as front-facing.

This will however cause your shadow volumes to shrink a bit, which may or may not be acceptable.

Another variation is to change the vertex-normals so that any vertex that touches a back-facing triangle receives zero lighting, which would keep your shadow volumes the same, but cause the shadow edge to grow a bit instead, as the shadow would be blurred into the lighted part instead of the light being blurred into the shadowed part.

PARTNERS