Well, I implemented my voxel bitmask class, and use it to create a shadowed/unshadowed value for every 1x1x1 meter area near a light.
Here's how it works :
For each light, if the light is directional, use the bounding box of the level's static world geometry.
Otherwise, use the bounding box derived from the point light's position & range.
Use the min of this bounding box to offset a 3d coordinate, which is iterated through the 3d volume defined by the box. For each 1x1x1 meter voxel, do 9 raycasts from that 3d point towards the light. If more half or more pass, consider the square lit, otherwise shadowed. Store the bit either way.
Then, at runtime, find which light bounding boxes touch the player's bounding box. For each of these lights, snap the player's bounding box to the light's grid, and sample a volume of the bit values, and count them up. I only use the upper vertical half of the box for the player, so triangles at his feet don't count as 'shadowed' when calculating his lighting.
Now for the slightly tricky part. Ideally we want to sample several spots on the grid, but we don't want the lighting to 'pop' too much as he moves across the map, so we'd like to do a trilinear filter, so that as he moves between a sample point in shadow and one that's not in shadow, his lighting increases relatively slowly as he moves.
This is a bit challenging, because we may have several sample points, and each sample point is a different distance from the player, and there may be many or fewer sample points, depending on exactly where in the grid he falls. For instance, a 1x1x1 meter dude may touch 2x2x2 cells or 1x1x1 cells depending on where he lands exactly, plus other combinations like 2x1x2. If you do a simple type of average, his lighting values will vary widely as he moves just a little bit.
So, the solution requires somehow weighting nearby sample points more, and far away samples less, and having all sample weights add up to 1, so the lighting is the right brightness.
This wasn't totally trivial to figure out, especially because using distance is sort of reversed in that a distance of zero should get the most weight.
The trick here is to use 1 / distance for each sample point. Then you need a renormalizing term to bring the total lighting to not exceed 1.
What you do is to sum up 1 / distance for all samples, and then do 1 / this total. This last reciprocal gives you the final weight value.
lighting = 0.0f;
sum = 0.0f;
for each sample point n
dn = max( 1.0f, distance from sample point to player )
sum += 1.0f / dn;
lighting += 1/dn * lit // lit equals ( 1.0 or 0.0 )
lighting *= 1.0f / max( 1.0f, sum ) // bring weight back to 1.0
You have to handle the distances being zero, and also, not to let the close distances get weighted too much, or the lighting is not smooth. Both can be handled by doing
sample_weight = 1.0f / ( max( 1.0f, dn ) );
When I last worked on this, I found several alternate schemes that worked out to the same math.
Anyway, here are the latest screenshots, one of the character in shadow, the other of him near an orange point light.
The next task is to apply this same idea for ambient occlusion. The level geometry has an ambient occlusion term stored in a channel of a lightmap texture. This is what gives the interior of the little blood pit a dark fringe around the edges. This looks great, but it looks wrong when a character in a dark area is lit too much.
Again, my old technique was raycasting, this time in a hemisphere about the player. This was a bit slow, especially around dense areas, and was a bit noisy as well. To speed it up, I made a raycasting routine that would treat all AABB leafs as solid, to avoid the ray/tri intersection tests, but this produced too many incorrect ray casts that would be quite distracting at times.
So, I will make a voxel grid for the level itself, calculated very similarly to the lighting, and sample that. But, that's for later on tonight or tomorrow.