I have a very large world, with a significant number of dynamic lights. Many of you have tackled this previously, so you know I am looking for novel ways to massively increase the number of shadow casting lights.

I'd like your advice on whether the following is possible/feasible/sensible, and whether anyone has done it before:

Rather than rendering the whole scene to a shadow map for each light, we instead rasterise the scene once, to a large voxel texture stored in projection space.

Then for each fragment, we march into the voxel texture to find the first intersection (our volume is in projection space, so this is a simple linear walk), and from that intersection point we raymarch to the position of each light (which need to be passed in projection space), stopping if we encounter a filled voxel.[/quote]
Pros:

• Cuts the render equation down from geometry * lights to geometry + lights (much like deferred lighting).
  
• We can use the voxel representation to compute an ambient occlusion term.

Cons:

• Requires a lot of GPU memory for the voxel texture.
  
• To allow for shadow-caster outside of the clip volume, we have to extend our voxel volume beyond the clip volume, and this costs us valuable resolution.
  
• The low resolution, coupled with the volume not being aligned with light-space, may mean a lot of aliasing.

We experimented with a similar algorithm in our current title, but only using the z-buffer instead of a full voxel representation. For each shadow-receiving pixel in the z-buffer (we use tile-based classification to quickly reject non-shadow receivers), we reconstruct it's position and trace a ray in the direction of the light source. Along the ray, we sample the z-buffer to see if a collision has occurred (with some 'fuzzy' metrics a'la SSAO - as if there's too much z-distance between occluder and occludee, it probably wasn't a collision).

Aside from the SSAOesque artefacts from only having a 2D scene representation (which this voxel approach would solve), and the fact that only casters within the clip-space unit-cube are considered, this gave us the same quality results as shadow-mapping for a much smaller cost.

To get soft shadows, we randomly jittered the rays slightly, and performed the "ray-tracing" on 1/4 of the required pixels and used a DSF/bilateral blur and upsample.

[edit] Trying to think of similar reserach to your quote, I'm reminded of "Interactive Indirect Illumination Using Voxel Cone Tracing", and IIRC the shadowed-reflections from EPIC's samaritan demo were achieved by pre-baking the static scene into a 3d distance field texture, which they could easily trace in any direction to detect light blockers.

We experimented with a similar algorithm in our current title, but only using the z-buffer instead of a full voxel representation. For each shadow-receiving pixel in the z-buffer (we use tile-based classification to quickly reject non-shadow receivers), we reconstruct it's position and trace a ray in the direction of the light source. Along the ray, we sample the z-buffer to see if a collision has occurred (with some 'fuzzy' metrics a'la SSAO - as if there's too much z-distance between occluder and occludee, it probably wasn't a collision).

Aside from the SSAOesque artefacts from only having a 2D scene representation (which this voxel approach would solve), and the fact that only casters within the clip-space unit-cube are considered, this gave us the same quality results as shadow-mapping for a much smaller cost.

To get soft shadows, we randomly jittered the rays slightly, and performed the "ray-tracing" on 1/4 of the required pixels and used a DSF/bilateral blur and upsample.

I think CryTek did something similar for Crysis 2...they called it "Screen Space Self Shadows" or something like that.

[edit] Trying to think of similar reserach to your quote, I'm reminded of "Interactive Indirect Illumination Using Voxel Cone Tracing", and IIRC the shadowed-reflections from EPIC's samaritan demo were achieved by pre-baking the static scene into a 3d distance field texture, which they could easily trace in any direction to detect light blockers.

Ah, nice. Not sure how relevant their use of voxel representations are to this, but it does suggest that the performance characteristics of voxel ray marching are reasonable.

I think CryTek did something similar for Crysis 2...they called it "Screen Space Self Shadows" or something like that.

I looked at some of their whitepapers when they first released that. It was interesting as an enhancement to SSAO, but I don't think they explored it much further than that.

Aside from the SSAOesque artefacts from only having a 2D scene representation (which this voxel approach would solve), and the fact that only casters within the clip-space unit-cube are considered, this gave us the same quality results as shadow-mapping for a much smaller cost.

I dismissed the 2D approach out-of-hand on the basis of the clip-space problem, without really noticing the depth issue. Ironically, the clip-space issue is probably the easier part to solve - you just increase shadow map resolution, and double the size of the clip-space cube you consider (i.e. regard the range [-2,2]). And increasing the resolution doesn't even hurt that badly in the 2D case (by comparison to 3D, at any rate).

The depth issue is the thornier part, and the reason I suggested voxels. I'm not sure we actually need a full voxel representation, though - perhaps it would be possible to store a set of intervals instead. Generating these on the GPU is trickier. Would probably need something along the lines of DX11's per-pixel linked-lists, and render back-faces as well during the voxelisation pass.

Actually, that doesn't sound that bad at all, but that would make the ray-tracing step significantly more tricky... Hmm, have to think more about this.

I write out the scene during the z-prepass as 1bit voxel into an unordered buffer, I address, it's width/2 | height/2 | 1024 and the view range is limited to 50m.

I use it for volumetric ambient occlusion, it's working quite fine. some maybe not obvious disadvantages

- faces orthogonal to the view do not contribute to the volume, which you can reproduce if you really want to see it, e.g. if you crunch on the ground at car-floor level, their "dropshadow" caused by the volumetric AO disappears.

-no early-z optimization is on, this causes quite some overdraw, but I dont want to add another separate pass yet, it sounds like I could render in 1/2th resolution, but then the volume in z-direction is really spares, oversampling 4x really closes most holes.

- just the clear of my unordered buffer is already in the ms area.

- another thing is, that linear tracing through space is not correct, it looks somehow correct, but you're actually in a bended space (I use a rescaled .w for the z index into the volume)

- it's a bit slow atm, fighting with 30fps on a GTX580

- I use atomicor, normal updates sometimes create a gap, but I plan to remove it and compensate those rare artifacts with a temporal solution (like some console games to for ssao)

good things are

- you actually dont see any aliasing during motion 99% of the time, even if you can, you just see it in a debug mode, combined with the scene it's perfectly fine looking (I wish I was allowed to show something )

- finally no ghosting like SSAO does, I know, that's obvious, but when you compare 1:1 it makes every gfx programmer smile

- it's very easy to integrate with dx11, you just set those buffer in addition to your rendertarget views for generation and you can extend your SSAO by sampling far distances rays in the volume, ssao is just a speed up, the quality with only the volume texture even for fine details (e.g. when using the normalmap for occlusion calculation, nut just the depth like the orginal SSAO)