Originally I wasn't even sure whether to post the question here - but why not. I do know the technique and how it works (which I'd like to elaborate in blog post - and use it as a reference for comparison), but so far I haven't seen any working example of this technique and neither saw it used in a game (I know there is one NVidia demo actually showing it - or a variant of it).
So I'm quite curious - has anybody here ever seen some real example of back-projection soft shadows?
Probably not, but i never understood the concept well anyways, so looking towards your blog post... 
What confuses me is for once to think about the pyramid from shadow map to lightsource, and secondly to think about the pyramid from sample point towards light - it's confusing. I do not really get the difference from PCF - maybe you manage to explain this better.
To me it's always seemed a bit silly, slow, and prone to lightleak. You could create some sort of acceleration structure out of the shadow map and in effect cone trace or raytrace through an SDF through it. But it's still going to lightleak. It's also going to be quite slow.
Frankly I'm just hoping whatever Ubisoft did for Far Cry 5 shows up at Siggraph or something soon. They've got variable penumbra sun shadows that seem very, very fast, have a lightshape (though I don't know why they chose a hexagonal light shape), look good, and again seem very, very, very fast. Fast enough to run on an Xbox One if I'm remembering right.
I still don't think my back-projection shadows code is correct (it sort of works, but needs improvement before it's article-worthy ... which is what is currently slowing down the shadows article I'm writing). Which is why I'm interested in some real implementation - there are just few papers not really going further into details.
For soft shadows (small area lights) are solved by penumbra size calculation and then using mip-maps to do smoothly blurred shadows (unlike PCSS which suffers of noise).
The problem are large area lights (hence backprojection), I could even use voxel representation of the scene and do cone tracing (which doesn't suffer of light leaking).
Currently for each light I allow user to pick from various shadows technique (from basic shadow map, PCF, PCSS, mip-map based penumbrae shadows, etc.). Adding back projection as option for large-scale area lights is a huge advantage for me - the problem seems to be performance (at least my current implementation is a no-go).
Maybe you have missed one of these:
http://casual-effects.com/research/Heitz2018Shadow/index.html
I would be interested in the idea to use regular shadow mapping but jitter the light position over frames, and using temporal reprojection / screenspace blur etc. accepting temporal lag. But that's lot's of effort to get working i guess.
8 hours ago, Vilem Otte said:I could even use voxel representation of the scene and do cone tracing
Have you tried this already? Might be good if you use AO to darken the shadow near contact.
But if you would support emmisive voxels, would you still need an additional aera light technique at all?
Unfortunately area shadows is basically an unsolved problem right now. Signed distance fields can be fast, but get expensive when animating a lot or getting too detailed. Cone tracing would probably be leaky and unstable for shadows. Right now there's just not a great answer.
But a good answer might be Moment Based Shadow mapping. It's the fastest/best way I know of to get a large filter radius in renderable time. There's hundreds of pages to read on the stuff, you can read that, anti-aliased moment shadow mapping, some other papers about improving lightleak a lot, etc. etc. etc. But the point is soft shadow mapping with a large filter is doable on the relative cheap with it.
Edit- AFAIR temporally supersampled shadows have been tried before, for anti-aliasing. But you'd have to re-do an entire temporal reprojection pipeline for each one to avoid obvious ghosting and etc. just like visibility sampling temporal AA. Probably not worth it, either in production time or milliseconds.
I've been actually experimenting with various shadowing every evening in past week - apart from the ones I've implemented.
So far the PCSS and PCMLSM (it's a PCSS variant where I use mipmaps to get more-smooth shadows using trilinear filtering) looks really good for small lights, but once you increase size it collapses (mainly because penumbra search is going to be wrong). Additionally PCSS suffers from noise, see the following:
PCSS - small area light
PCSS - large area light
In PCMLSM I got rid of PCSS noise, and it looks really good, for small-size area lights:
PCMLSM - Small area light
PCMLSM - large area light
I intentionally didn't do any attenuation at all. For comparison ISM-like approach with lots of small lights instead:
This looks by far superior to previous ones, yet the fillrate is huge (notice - you can see the actual shadow maps in the virtual shadow map to the right). This is quite close to the actual result I'd like to see (for large area lights).
Cone traced shadows are interesting but most likely a no-go simply because of the resolution of the voxelized scene - I just quickly played with cone size for the ambient occlusion and cone direction (to point towards the light), and you can clearly see that while it is generating shadows - they're not well-defined and suffer of light leaking (due to F.e. empty interior voxels of the sphere).
Using SDF instead could be good, but they will still suffer with lower resolution. Not to mention problems with animated models. I'd personally like to try it - yet it might be quite challenging.
Back-projection seem quite promising for such scenarios (yet the only working example which is from NVidia doesn't really work well with such scenario unless you invest large amount of steps in it - which again ends up in slowness) - which might actually render it useless in the end.
I'm now trying to implement the prototype I have into the actual editor so I share some nice screenshots and finish up the article (which is still in-progress), yet I do have some problems with different light types (point vs. spot), and generally the actual projection which seems incorrect to me.
If you're going for that big of an area lightsource then you're basically doing global illumination, so I'd skip looking at direct shadowing and try out voxel cone tracing/Signed distance field tracing.
Actually SDF tracing is probably what you're looking for. Unlike cone tracing you can check the sign to see if your inside a mesh, so your empty voxel problem is solved.
http://advances.realtimerendering.com/s2015/DynamicOcclusionWithSignedDistanceFields.pdf
Epic did an experiment with using this to shadow VPLs. It worked out pretty well, until they abruptly cancelled all experiments as such: https://forums.unrealengine.com/development-discussion/rendering/40411-the-state-of-distance-field-gi-in-4-8
6 hours ago, FreneticPonE said:Epic did an experiment with using this to shadow VPLs. It worked out pretty well
CryEngine has this feature running IIRC (voxelization but no SDF), but their manual pages seem to be down at the moment.
So, I implemented sort-of backprojection that works in stochastic manner (requiring a lot less steps) - as when I did it per-original implementation I was getting up to 4000 iterations per pixel, which is a no-go for performance.
The original idea of back-projection is that we have an area light of specific area A, and we project back texels from depth map calculating how much you overlap this area and subtract from it. This will end up in severe light leaking and really bad performance (simply because there is too many texels to walk through). To improve performance you can have mip-chain for shadow map (with min-max trees stored in there), which can significantly improve performance - yet it seems way too complex to me.
Instead, I tried to implement stochastic approach to back-projection. Which is simply performing ray-casts against depth map (again min-max trees for shadow maps can help here with performance).
Starting off with picking N (in showed cases 32) samples on area light, and performing a ray cast against depth map whether we find any intersection on way towards light. The results seem quite promising, yet suffers from the same problem as PCSS - with growing light size, performance drops - see following 4 images (and notice the fps in top-right corner). This was with brute-force ray-casting with 64 steps - I didn't use the hierarchical representation I have to my advantage (although that will complicate traversal further!).
I still have some small bug in the code that over-does my internal umbra (I intentionally darked penumbra part - as it looks like only outer penumbra is calculated properly, while the inner one is always at full shadow strength - even where it shouldn't be), so I'll check whether something can be done with that. Other than that while this is usable for larger area lights, it still doesn't deliver the results I'd expect (or at performance I would expect).
