Thanks again to you all, for your helpful feedback and suggestions.
@mv348, do you use a high-poly mesh for rendering, but a low-poly mesh for shadows? Or do you use the same mesh for both?
Using the same for both. Its relatively low poly.
So I tried implementing what Erik Rufelt suggested:
You can fix it by, when you extrude the edges between back and front-facing polygons, instead of using the triangle plane normal to determine facing, looping over the three vertex normals in the triangle and checking if any of the vertex normals face the light. If at least one normal faces the light, then consider the triangle front-facing.
This will however cause your shadow volumes to shrink a bit, which may or may not be acceptable.
Pretty much did exactly what you said. Definitely fixes the self shadowing issue. However, the shadow shrinking produces a shadow with a jagged, per-polygon edge. So ironically it really just moves the same jagged artifacts off the self-shadows and puts them on the regular shadows. :\
@Hodgeman, you said:
As long as you're only extruding truly back-facing edges, this artefact shouldn't occur, or more specifically, it should only occur in areas that are un-lit, so you can't notice.
Are you suggesting the same technique as Erik? What do you mean by a truly back-facing edge?
@Kryzon, you said:
Fortunately, after some research I got to know the "Reverse-Extruded Shadow Volumes" technique by Renaldas Zioma.
In simple terms, to create the volume, instead of extruding the back faces and leaving the front faces in place, you do the opposite.
I fail to see how extruding the front-faces instead makes the self-shadowing problem any better. Don't have time to read through and digest a paper right now. Could you give me a few more details on how they resolve the self-shadowing issue? I already have ambient shadows in place. (which I assume means, only ambient light is rendered in shadowed pixels)