Shadow map flickering when lights move

Which shadow mapping technique are you using?

For large outdoor scenes, PSSM (Parallel-Split Shadow Map) is recommended. Assassin's Creed II combines it with VSM (Variance Shadow maps) to hide even further the subtle (but annoying) flickering of day-night cycles.

IIRC, Burnout Paradise uses VSM too to hide day-night cycle flickering.

So... which technique are you using?
Cheers
Dark Sylinc

Another technique that is better than the Variance Shadow Map (VSM), is Exponential Shadow Map (ESM). ESM's are about 10% faster, use half the storage of VSM's and light bleeding for the most part is eliminated.

Read up on which ever though. VSM's have lots of examples, to learn from. There are no examples to learn ESM's from, but VSM's and ESM's are so similiar, a change from one to the other literally, takes about 10 lines of code to change.

Too many acronyms . . .

With games becoming more and more dynamic, there is pretty much nothing that you can do except increase resolution and filtering.

Before when everything was static you could use Stable Cascaded Shadow Maps: pixel centers are aligned in every frame and shadow map segments ware not rotated. Problem with this is low shadow map utilization, but it is was worth it because there was no flickering.

But now, when we have a lot of vegetation that even slightly moves this becomes useless. Example games are Crysis and Sniper Ghost warrior (they use Stable CSM even though they have dynamic vegetation - which is not very smart because "Stable" doesn't work in this case). In new Crysis, usage of Stable CSM is justified because there isn't much vegetation - it happens in the city where most of the geometry is static (step back if you ask me).

I recommend using new shadow mapping approaches such as CAMERA SPACE SHADOW MAPS that completely utilizes shadow map so you can have achieve better results with less work (only one pass and only one SM). There will still be flickering, but you can use much smaller shadow map and less filtering to achieve better results.

Quote:

Original post by Matias Goldberg Which shadow mapping technique are you using? For large outdoor scenes, PSSM (Parallel-Split Shadow Map) is recommended. Assassin's Creed II combines it with VSM (Variance Shadow maps) to hide even further the subtle (but annoying) flickering of day-night cycles. IIRC, Burnout Paradise uses VSM too to hide day-night cycle flickering. So... which technique are you using? Cheers Dark Sylinc

CSM. At least, I think so - I'm a little unclear on the difference between CSM and PSSM. Basically, I split the light frustum in three and use a map per split.

I did initially also use VSM, but dropped it because it resulted in pretty bad light leaks and was actually slower than vanilla SM, even with PCF. I'll have a look at ESM, as smasherprog suggests. But I'm not sure how it would help with this problem: even if you use a large blur kernel, wouldn't that just "spread out" the flickering, rather than making it less noticeable?

Quote:

Original post by myers CSM. At least, I think so - I'm a little unclear on the difference between CSM and PSSM. Basically, I split the light frustum in three and use a map per split.

I assume you say CSM as in cascaded shadow maps (and not Convultion shadow maps)
PSSM is a type of CSM.
One key aspect is that PSSM bases the position of the each shadow camera scientifically (with a simple lerp that weights a linear with an exponential formula), where's other CSM implementations determine the cascades/split points empirically or with other formulas.

Quote:

Original post by myers But I'm not sure how it would help with this problem: even if you use a large blur kernel, wouldn't that just "spread out" the flickering, rather than making it less noticeable?

Yes, it might, but you sure alleviate the problem (and sometimes a lot).
Imagine just two pixels, one black (shadowed) another white (lit). When you move the camera or the light, the two pixels shift their colours due to sampling problems (the white pixel becomes the black one and vice versa)
If you use filtering, both pixels will be grey. Furthermore if both are equally gray, even if they constantly switch places due to sampling imprecisions, they would still have the same colour aka. no flicker at all.

That was a simplified example, but on the big picture, errors and flickering get blended with correct pixels producing smoother results.
Of course very large kernels will introduce other problems, such as pixels that should be fully lit becoming slightly shadowed.

Cheers
Dark Sylinc

See the section "Moving the Light in Texel-Sized Increments" in Common Techniques to Improve Shadow Depth Maps

Hope this helps.

Quote:

Original post by Matias Goldberg Quote: Original post by myers But I'm not sure how it would help with this problem: even if you use a large blur kernel, wouldn't that just "spread out" the flickering, rather than making it less noticeable? Yes, it might, but you sure alleviate the problem (and sometimes a lot). Imagine just two pixels, one black (shadowed) another white (lit). When you move the camera or the light, the two pixels shift their colours due to sampling problems (the white pixel becomes the black one and vice versa) If you use filtering, both pixels will be grey. Furthermore if both are equally gray, even if they constantly switch places due to sampling imprecisions, they would still have the same colour aka. no flicker at all. That was a simplified example, but on the big picture, errors and flickering get blended with correct pixels producing smoother results. Of course very large kernels will introduce other problems, such as pixels that should be fully lit becoming slightly shadowed.

Or (more glaringly, I think) pixels that should be shadowed becoming lit. I hope to get a chance to implement ESM this week and report back, but bad experiences have made me generally wary of covering up artifacts simply by increasing blur size.

If this is the standard approach, though, I'll certainly give it a try. Someone mentioned Crysis - I believe they use VSM, so I'll have to check that out and see how shadows cast by moving vegetation look.

Quote:

Original post by 2square See the section "Moving the Light in Texel-Sized Increments" in Common Techniques to Improve Shadow Depth Maps Hope this helps.

That discusses a slightly different issue, though, which is achieving stability with CSM when the camera moves, not the light.

Reviving an old thread. I was hoping there would be some good answer in the end, but it doesn't look like it :-).

I just encountered this problem with my current project when I started having the day/night cycle change automatically. I'm going with roughly 24 minutes of real time per game time, and the shadows "swimming" as the sun moves is very noticeable.

I tried implementing the blending the O.P. mentioned. e.g. I update the light direction every 5 seconds and gradually fade the new shadows in. Obviously this comes at a pretty big performance cost, but I mitigate it somewhat by have a "band" that moves across the screen that is a fraction of the screen width. Only the region in there is undergoing blending, so I save a good amount of texture fetch bandwidth (I still have to render two shadow maps every frame though). The effect is a little strange though, and I'm not sure I'm satisfied with it.

Another option would be blending things in pixel by pixel or block by block using two passes and the stencil buffer. I'm not sure if that would look good though.

I looked at Fable 2 which has day/night cycles to see how it was done there. They have very soft penumbras, which I think helps a lot. Their shadows are also not very dark (lots of ambient light). And it looks like only some objects cast shadows. Notably people and vegetation, both of which are constantly moving - so that's another thing that helps. I walked around a bit but didn't see very many static objects casting shadows outside. Looking closely at some large tree trunks, you do see the shadows shimmer constantly as they move with respect to the sun. But it's fairly well hidden by the lack of sharp shadow definition.

Anyone have any other ideas? (please note, this is about artifacts when the light source moves, not the camera - a few posters seemed to be confused about that).

As an FYI, I wrote up a summary of the things I tried and the results here:

http://mtnphil.wordpress.com/2011/10/31/shadow-maps-for-moving-light-sources/

The original poster mentioned reducing texture memory usage by rendering both "previous" and "current" shadow maps into the same texture. This is pretty straightforward if you adjust the color write channels when rendering (I describe what I did in the blog post above).

We have dynamic day/night in our upcoming release, time of day is always running, so shadow edge jittering is real problem. Simple solution - we only update the light vector about once every 10 seconds or so.

We have dynamic day/night in our upcoming release, time of day is always running, so shadow edge jittering is real problem. Simple solution - we only update the light vector about once every 10 seconds or so.

And you don't find the "jump" is too noticeable every 10 seconds? I'm working with a top-down-ish view, so it may be more of a problem for me, since the shadows are always plainly visible.

Have you tried grid snapping your shadow projection matrix so that it only ever moves in exact texel increments? Fixes much of the swimming of static shadow geometry when the lights move.
Rotation is a different matter altogether though, that trick only really works for translations.

Have you tried grid snapping your shadow projection matrix so that it only ever moves in exact texel increments? Fixes much of the swimming of static shadow geometry when the lights move.
Rotation is a different matter altogether though, that trick only really works for translations.

As pointed out before, that technique is great for camera movement, but it doesn't help when the lights move. Once a light moves, the resulting shadow map is completely different (whereas when the camera moves, you can move your shadow map in whole texel increments so it is an exact copy of what it was before, just shifted over a pixel).

Specifically, the problem we're trying to solve is for a directional light representing the sun/moon as it moves across the sky.

I can only recommend making sure you've got good frustum and occlusion culling built into your camera and your shadow rendering algorithm so that you're not drawing too much mesh so that you can use a higher resolution.

I would also recommend the use of shadow meshes for shadow casting objects. That is you have a mesh of character that is rendered in the diffuse pass, but then you have a low res mesh of that character that is rendered in the shadow map, the shadow mesh would be really low poly, it may even look polygonal on the ground.... You may even use rigid skinning. It comes down to what you want to get out of it, smooth round shadows the shimmer, or slightly polygonal shadows that shade evenly. At the very least your shadow mesh would be positions only, no normals, tex coords, mapping coordinates etc to pass through the buffer and ignore, so that could speed up the rendering time itself.

But you've probably already done all that, all I'm sayin' is check your bases.

Oh, you're using shadows from a sun object. Don't use a texture map for the shadow map then, use a shadow map that's baked into the vertices of the mesh.... If you use a color channel you can bake 4 values for shadow based off the sun into the mesh, and use dot products based off sun direction to interpolate them to approximate the shadow cast by the sun. Interpolate them any way you want, you can probably pre-compute the dot product of the sun direction vector before any rendering starts any way, it all comes down to how you bake the data into the mesh to determine how easily you get that back.

And if you're looking for sharp edged shadows for you trees or whatever being cast by the sun, use that as a secondary solution, make a camera frustum with a far plane that is pretty close, intersecting with a frustum that starts on the high egde of the intersection of the camera and the light source, and ends on the low intersection with the camera and the light source, and just do shadows for a much smaller amount of geometry, I would still use the baked shadows for stuff far away though, no point having trees at one section of the map suddenly go from light to dark as the shadow map kicks in.

In your pixel shader that incoperates your shadow rendering, I think you should be able to use the w (if I remember it correctly) value of the projected position, with a scale and a clamp, to deal with the interpolation from baked shading to shadow map. Just use a scale mod to stretch it out and a clamp to keep it in the 0 to 1 range. And use the resulting value to interpolate between the baked shadow value and the value on the shadow mesh. The shadows should appear to fade in over a distance.

The key is that the baked stuff is just to give the correct sort of shading to an area, you might want to fiddle with your ray traces in your baking application to set the shading of a surface dependent on the proportion of that surface unobstructed from a general direction.
In order to know when you should be using the shadow rendering pixel shader, you should use a BSP ( http://www.gamedev.n...ics/bsp-tree-r2 ) and when the current object in your BSP passes a certain depth limit ( if you're going ftb) switch the shadow rendering shader off, or goes under a certain limit ( rendering btf ) switch it on . If you're not using (don't have time to implement) a BSP, just go:
if ( pos.w > 0.9 ) {
// Shadow map lookup stuff here
// Interpolation of shadow map and baked map, apply to output color
}
else
{
// Use baked map
}

Keep in mind that the baked map adds more data to the vertices which is never a good thing. If you have a 2 value mapping coordinate channel and your sun is only rotating along one axis (around a cardinal axis or custom axis ), use 2 more values in the tex map to do the baked shading. I think it loads into the pixel shader as 4 components for performance reasons anyway (that was just a guess)

Or, if your terrain has normal maps anyway for gourad shading, fob those off and replace the normals with a 4 component value with two components for shading of the surface based off light direction and two components for shadows and interpolate your shading values based off the dot product of the light vector against the horizon as well as your shadow 'shading' values.

Only problem is if you have a valley or something like that so it will look dark even in the middle of the day, at which point use 4 components that are tied to 2 cardinal axis around the rotation axis of the sun and use the dot product of the sun against the horizon and the dot product of the sun against the vertical for your shading calculations

I can only recommend making sure you've got good frustum and occlusion culling built into your camera and your shadow rendering algorithm so that you're not drawing too much mesh so that you can use a higher resolution.

It's certainly not perfect (there is a good amount of the shadow map render target that is "wasted"), but it's about as good as I can do.

I would also recommend the use of shadow meshes for shadow casting objects. That is you have a mesh of character that is rendered in the diffuse pass, but then you have a low res mesh of that character that is rendered in the shadow map, the shadow mesh would be really low poly, it may even look polygonal on the ground.... You may even use rigid skinning. It comes down to what you want to get out of it, smooth round shadows the shimmer, or slightly polygonal shadows that shade evenly. At the very least your shadow mesh would be positions only, no normals, tex coords, mapping coordinates etc to pass through the buffer and ignore, so that could speed up the rendering time itself.
[/quote]

The shimmering happens even for low-poly objects (e.g. even a simple cube), so I don't think reducing the poly count for things will help. It's certainly good advice to reduce the size of the mesh vertices though, and the amount of data passed out of the vertex shader.

Oh, you're using shadows from a sun object. Don't use a texture map for the shadow map then, use a shadow map that's baked into the vertices of the mesh.... If you use a color channel you can bake 4 values for shadow based off the sun into the mesh, and use dot products based off sun direction to interpolate them to approximate the shadow cast by the sun. Interpolate them any way you want, you can probably pre-compute the dot product of the sun direction vector before any rendering starts any way, it all comes down to how you bake the data into the mesh to determine how easily you get that back.
[/quote]

It's sounds like maybe you're talking about baking information into the vertices about how they get shadowed by adjacent geometry depending on the sun's position? That's an interesting idea, but I think I have too much moving geometry for that to appear convincing.

And if you're looking for sharp edged shadows for you trees or whatever being cast by the sun, use that as a secondary solution, make a camera frustum with a far plane that is pretty close, intersecting with a frustum that starts on the high egde of the intersection of the camera and the light source, and ends on the low intersection with the camera and the light source, and just do shadows for a much smaller amount of geometry, I would still use the baked shadows for stuff far away though, no point having trees at one section of the map suddenly go from light to dark as the shadow map kicks in.
[/quote]

In my particular case, I'm using a 45 degree "overhead" view, so there aren't really any far away objects. Everything is kind of medium distance. I think cascading shadow maps would be another solution to the issue you're talking about, but I don't need them in my scenario since everything is roughly the same distance away.

The key is that the baked stuff is just to give the correct sort of shading to an area, you might want to fiddle with your ray traces in your baking application to set the shading of a surface dependent on the proportion of that surface unobstructed from a general direction.
[/quote]

I'm already using ambient occlusion baked into the vertices of many of my scene objects, but that's a separate issue from the shadow maps.

Anyway, thanks for the very detailed post!


I do actually have another idea for how to address the shimmering from moving lights. I would blend between two light maps (which are greyscale texture that is the result of doing the shadow map comparisons) for two different sun positions. To avoid additional rendering cost you could render each one on alternate frames (this would produce artifacts for fast-moving objects though). For a deferred rendering engine (which is what I'm using now), a separate light map would mean an additional render target. But for a light pre-pass renderer this might be a more natural fit. If I get around to trying it out I'll post back with the results.