Sign in to follow this  
the_xmvlad

Extended PSM (research paper, demo, code)

Recommended Posts

Extended PSM (XPSM) provides practical solution for real time shadows rendering. Distinguishing features of this method from other shadow map reparameterization techniques is: * high quality shadows, based on suboptimal projective space parameterization * complete artifact reduction with correct post perspective space Z bias * stability, free from singularities * easy integration into any shadow map based engine * patent clearance Research paper, demo, code at http://xpsm.org XPSM screenshot XPSM screenshot [Edited by - the_xmvlad on August 12, 2007 2:31:42 AM]

Share this post


Link to post
Share on other sites
Wow, I've only checked out the demo but that looks REALLY great. Especially the long stretched shadows with the light source being very low at the start of the demo. Could this be the holy grail of shadow mapping?

Just one small question, maybe I overlooked it but is there an fps counter missing?

Share this post


Link to post
Share on other sites
wolfganw:
thanks ) yes, for any reasonable case XPSM suboptimal, you can just check the paper for complete analysis. In terms of shadows resoluton XPSM is very close to TSM. But doesn't have usual heavy artifacts of TSM produced by problems like: z acne, z bias non-linearity, etc, and patents. ) Another very good XPSM feature is absolutely correct(per-pixel) Z bias, that completely reduce "visual crap" associated with usual "high perspective space z bias non-linearity" in other methods.

sorry, no fps counter, XPSM is just parameterization technique and have exactly same perfomace as standart shadow maps.

Share this post


Link to post
Share on other sites
After trying to demo, I have to say this is not particularly impressive, there is a lot of distortion at certain camera angles, and the resolution chages are very apparent, not any better than regular perspective shadow mapping. I prefer lower res but static shadows that dont pop and distort, especially when up close.

Share this post


Link to post
Share on other sites
Matt Aufderheide:
it is "miner's lamp" case, in which any parameterization method (LiSPSM/TSM/PSM/etc) reduces to standart "low res" focused shadow maps. And of course any dynamic shadows is just incomparable with perfect static light maps.

Share this post


Link to post
Share on other sites
Quote:
Original post by the_xmvlad
Matt Aufderheide:
it is "miner's lamp" case, in which any parameterization method (LiSPSM/TSM/PSM/etc) reduces to standart "low res" focused shadow maps. And of course any dynamic shadows is just incomparable with perfect static light maps.


What he means by "static" is that you don't see the shadow texels flicker when the camera moves. Not static as in lightmaps.

I made lots of research on shadowing techniques recently (particularly TSM and LiSPSM) so I had a look at your demo and paper. It looks pretty nice, especially as it fixes a few of the problems remaining with TSM (which I already found to be a nice improvement over LiSPSM). But it's far from being the "Holy Grail" of shadowing IMO. Just yet another little improvement.

The worst case (light direction parallel to view dir), aka the "miner's lamp" case as you call it (first time I hear such term), is what prevents all those techniques from being practical IMO. They're fine in specific cases when you have a fixed viewpoint (like in real-time strategy games), or when the range at which you want to apply shadows is a few hundred meters in your typical scene, at most. Unfortunately I'm still looking for a solution that can apply dynamic shadows to an entire scene, which in my case can have an horizon of up to tens of KM, with a constant/good quality.

I think I will continue my research by falling back to simple uniform shadow maps, but adapted to the view frustum, and use a serie of them (like in cascaded shadow maps, or parallel-split shadow maps). It's easy to implement, but I think they'll give the best trade-off between quality (lack of artifacts) and performance.

Share this post


Link to post
Share on other sites
Ysaneya:
oh, i really talking about suboptimality of XPSM parameterizaion, that is more like "holy grail of parameterization", sorry for misinterpretation. CSM and PSSM is great techniques but have they problems, too: different resolution in each split, that produce some artifacts in places where different splits jointed. And parameterization methods can be applied to CSM/PSSM splits for resolution improvement, as to a standart shadow maps.

i know at least one game where TSM was applied to shadow, large scale environment - S.T.A.L.K.E.R and with "bluring" (soft shadows) it looks really good.

Share this post


Link to post
Share on other sites
Hi,

this paper looks quite interesting to me. I decided to try to integrate this algorithm into my engine, but I face some issue.

First I am not sure what is the ViewMatrix you use: is it the main scene view matrix (normal view) or the view matrix from the position of the light ?

The other issue I have is that my engine is using a scene graph approach, so I don't have access to individual objects. So my approach is to find the intersection of the bounding box of the casters with the view frustum (and extend it a little bit to include casters outside the frustum that produce shadow inside this frustum).
OK, so once this is done, I transform the 8 points of this box into the View Space (which I am not sure which it is - see above). This replace the step 4 of your algorithm.
Steps 11 to 14 also had to be modified because I only have this single bounding box.

However, I don't manage to make it works. And I have no idea what the problem is.

What I don't understand is, if the view matrix is the main camera view matrix, the scene is rendered from the main camera point of view and not the light point of view? Is that what happens? Usually, using directional lights, I render the scene from the light point of view (which basically is "bounding box center - bounding sphere radius * lightdir)..

Also, I don't understand this code line as well:

viewLightDir = -viewLightDir;

why do you negate the light dir ? I haven't found any mention of that in the paper...

thanks for your help!

[Edited by - gjaegy on August 17, 2007 4:47:05 AM]

Share this post


Link to post
Share on other sites
This is light space definition from paper: "In the light space the light direction is parallel to Z axis and the viewer origin translated to (0, 0, 0)". Simplest way to get into light space is:
1. take view space (usual camera space)
2. transform light vector into view space Lview
3. build LightSpace = Look At ((0,0,0), Lview) matrix, that aligns light direction with Z axis (look at Figure 2 from paper). so (View * LightSpace) matrix transform from some space (usually world space) into light space.

Next, you need to transform all you bounding volumes points (it don't seems optimal, to use box-frustum intersection but at least should work) into LIGHT SPACE, let's refer to it as LS Bouding Volume.

To process forward, you need to determine "warping effect"(fix projection vector direction and length). So this is done in two steps:
a) find direction of the projection vector
1. Transform camera view vector into light space. (camera direction in view space) - V = (0, 0, 1) * LightSpace
2. Project V to XY plane and normalize, that completely determine projection vector direction. unitP = V.xy / |V.xy|

b) find length of the projection vector
1. Project all points of LS Bounding Volume (that is already in light space) to unitP vector and find minimum. That is just: minProj = [for all Points in LS Bounding Volume] (unitP.X * Points.X + unitP.Y * Points.Y)
2. Compute maximal bound of the projection vector. maxLengthP = (epsilonW - 1) / minProj. (if you have separated Receivers\Casters more optimal version can be used look at paper)
3. Compute optimal projection vector length: lengthP = coef / cos(gamma),
cos(gamma) = unitP.x * V.x + unitP.y * V.y (figure 4)
4. Clip optimal length with maximal length to avoid singularities. (3.5 from paper)

Having projection vector it is possible to make transformation into this suboptumal "warped space" (3.4\3.5) (post projection light space or just PPLS). Then PPLS Transform = Projection * ZRotation - transformation from the light space into this "warped space" (PPLS). So we need to take all points of LS Bounding Volume and apply PPLS Transform to it (+ divide by w component, to put all points into real space, after transformation). Let's refer to it as PPLS Bounding Volume (bounding volume points transformed into "warped space").

Next, we need to transform part of this "warped space" into unit cube (device normalized coordinates). So we just build AABB in PPLS over PPLS Bounding Volume points. From this AABB linear basis can be easily constructed, and then inversion of this basis is transformation into unit cube. (UnitCubeTransform)

In last step simple transformation applied to transform unit cube into device normalized coordinates. And finally you need to combind all this transformations: XPSM Transform = View * LightSpace * PPLS Transform * UnitCubeTransform * NormalizedSpace. + you need to check left\right hand cordinate system, row-vector\column-vector math, issues if you using OpenGL.

and negotation to viewLightDir applied just because in demo light vector look from the light source, and in paper assumed that light vector look at the light source. i hope this will be helpful to resolve issues.

Share this post


Link to post
Share on other sites
Hi Vladislav,

first thanks for your quick answer.
I guess I might be stupid, as despite your very clear explanation I don't manage to make it work.

Actually I am using a combination of cascade shadow maps and XPSM; maybe the problem comes from that (i had this combination working for TSM before)? I split the frustum into 4 parts, each one being rendered in one quarter of the shadow map texture.

First, I am still not sure whether I have to negate the light dir or not. My light direction is like in your code ((0,-1,0) when sun is vertical) so I should have to negate. However, when I do that I get this (sun is vertical, no bias for the moment):

http://g.jaegy.free.fr/temp/xpsm01.jpg

look at the top left corner of the bounding box, gears of aircraft are visible, which means 1/Z is rendered to shadow map instead of Z...

When I don't negate I get this:

http://g.jaegy.free.fr/temp/xpsm02.jpg

Also, I found out that I get a good result only if I reduce the XPSMCoef to a very low value (0.0001) which is, in my opinion, not normal. Any idea? When the value get higher, the shadow moves to the back until it disappear...

Finally I found out that the fact I use the frustum/caster bounding box intersection instead of what you explain in your paper cause issue, as some parts don't get rendered into the shadow map, so I will have to check this part. I am not sure however what should be rendered in each quarter of the Cascade Shadow Map.

thanks again for your great support !

Share this post


Link to post
Share on other sites
oh, you are right, light direction should point from the light source, that is (0, -1, 0) is correct. (and reversed in original demo)

i think again, about this casters bounding box \ frustum intersection, it really incorrect. (due to back casters, etc). you need, take Casters as you objects bounding box points and Receivers as frustum points. and proceed as described in paper.. ie, project this Receivers and Casters to unitP, find minCastersProj, and minReceiversProj, calculate maxLengthP = (epsilonW - 1) / max(minCastersProj, minReceiversProj).. project Receivers and Casters into PPLS, build two AABBs, and find their geometrical intersection. (look at paper/demo) + it seems(just test at first) you need to apply W clip when transforming into PPLS (look TransformWithWClip from demo)

And it really normal when you get small Coef value, they generally depend on scene scale, near\far objects tradeof, and in usual case should be very small. + try to tune epsilonW (normal value ~0.85)

PS: if you get individual casters\receivers points, or just two separate casters\receivers bounding boxes, then it can produce much better focusing.
PSS: interesting to see some screenshots when you complete XPSM integration! )

Share this post


Link to post
Share on other sites
Vladislav, you were completely right; I implemented your suggestion, using the 8 frustum points for receivers box. I also traverse the scene graph to get the sub-meshes that are casters, and proceed the same way as you did in your demo.

And the result is completely...great !! Still completely unoptimized, no bias as well, but working. and compared to TSM, the blur kernel size (or scale of the shadow map texture sampling disc) doesn't change depending on the view/light direction changes. I guess the bias as well. it really behave in a much more "constant" way compared to TSM.

http://g.jaegy.free.fr/temp/xpsm03.jpg (actually nothing fancy at all).

I will continue the implementation during the week end or next week, and then send you some nicer pictures.

By the way, I still have 2 question:
- in your demo you seem to only use a constant bias. Don't you need any slope scale bias ?
- what are the role of EpsilonW, and Coef ? I didn't understand completely to be honnest...

Now, I am pretty sure adding VSM to this CSM+XPSM combination will produce very nice results...

Have a nice week end!

Share this post


Link to post
Share on other sites
Quote:
Original post by gjaegy
Now, I am pretty sure adding VSM to this CSM+XPSM combination will produce very nice results...

Yeah I'd like to see that too - seems like it would be quite nice. Even CSM+VSM looks great, so adding in a nicer per-split projection warp would probably help even more.

I also would like to mention that the whole "static projection" thing where people go through a large amount of effort to make sure that projected texel locations don't change sub-pixels in camera space is a bit odd... at that point you've basically given up on proper filtering, which totally isn't necessary! As I demonstrated in my D3D10 VSM demo earlier this year there's no need for any of that trickery when the shadow map is properly filtered - you just get smooth, filtered, anti-aliased shadows with edges that flicker neither when the camera moves, nor the light (assuming a reasonable projection and resolution of course, but even a few 512^2 CSM/PSSM slices are sufficient).

Anyways I haven't had a chance to dig into this particular technique, but it seems like it could be a good compliment to frustum splitting (CSM/PSSM) and filtering (VSM/proper PCF) algorithms.

Share this post


Link to post
Share on other sites
Quote:
Original post by AndyTX

I also would like to mention that the whole "static projection" thing where people go through a large amount of effort to make sure that projected texel locations don't change sub-pixels in camera space is a bit odd... at that point you've basically given up on proper filtering, which totally isn't necessary!


Are you serious? Excessive filtering might cover subpixel flickering, but that's only practical from a researchers point of view. At the moment, no graphics board can handle more than a couple of statically spreaded shadow samples in real game scenes, subpixel alignment is absolutely neccessary for these scenarios. Sadly, several games don't do this, and it is always criticised in the game's reviews.

A technique that prevents subpixel alignment is a no go for me. It's another reason to avoid all those shadow mapping methods based on perspective warping of the shadow map. The primary reason is the fallback to uniform shadow mapping at certain light angles in relation to the camera.

Bye, Thomas

Share this post


Link to post
Share on other sites
Quote:
Original post by Schrompf
Are you serious? Excessive filtering might cover subpixel flickering, but that's only practical from a researchers point of view. At the moment, no graphics board can handle more than a couple of statically spreaded shadow samples in real game scenes, subpixel alignment is absolutely neccessary for these scenarios.

This is totally untrue on the latest hardware (GF8's), as I showed in the VSM demo. With a decent splitting scheme (ex. PSSM), and ideally VSM + MSAA, and potentially a small minimum filter width (4x4 blur with VSM lets say - note that even 50x50 or more blurs are completely fine performance-wise with VSM, albeit overkill) you can easily get enough resolution to hide all projection movement even in large scenes. Warping techniques like this only improve the efficacy of each split.

Note that MSAA does make a huge difference here, just as it does for normal rendering. A 512x512 VSM with 4x MSAA looks significantly better than a 1024x1024 (and even 2048x2048) standard shadow map w/ PCF since it's a rotated grid, etc. Add in CSAA, use 2-4 1024x1024 splits and you're talking about sub-pixel resolutions that far exceed what's necessary even for today's high resolution displays. And yes, this is quite feasible for modern, fully dynamic scenes on GF8's for instance, which means that it'll be feasible for games in the near future.

Anyways I'm not saying that sub-pixel alignment is useless, it's just too inflexible and it's really "giving up" on actually getting enough resolution where you need it. Shadow edges flickering is unacceptable to me in *any* instance, including objects moving, lights moving and certainly the camera moving too - but I'm not happy solving just the latter.

If you doubt these claims, I do encourage you to check out the demo that I posted earlier this year on Beyond3D (source code and lots of info in GPU Gems 3, now available!).

Share this post


Link to post
Share on other sites
That's why I added "only practical from a researchers point of view". While I agree with you that achieving a screen subpixel shadow resolution is the best way to solve all aliasing artifacts, a game aiming for a Gf8800 is going to die a quick death at the market. This will be feasible in maybe some years from now, propably starting with the next generation consoles as consoles seem to set up the technical standards these days. For today, it's a nogo, and that's why I said I refuse to use any shadow mapping technique that does not allow subpixel alignment of the shadow maps.

Bye, Thomas

Share this post


Link to post
Share on other sites
Quote:
Original post by Schrompf
While I agree with you that achieving a screen subpixel shadow resolution is the best way to solve all aliasing artifacts, a game aiming for a Gf8800 is going to die a quick death at the market. This will be feasible in maybe some years from now, propably starting with the next generation consoles as consoles seem to set up the technical standards these days. For today, it's a nogo, and that's why I said I refuse to use any shadow mapping technique that does not allow subpixel alignment of the shadow maps.

Fair enough, although I'd argue that anyone starting development on a game today could consider DX10-class hardware a reasonable target.

Share this post


Link to post
Share on other sites
Hi,

I am struggling with the visibility culling part. Generally, I generate frustum planes (from a camera definition, a view-proj matrix or whatever, transforming back the unit cube into world space thrgouh the inverse view-proj matrix) and use these planes to determine objects that are completely inside, completely outside or partially inside the view frustum.


However, because of the special projection of this algorithm, this approach doesn't work anymore. Does any one has a suggestion?

An idea would be to use the 3D box representing the focus region basis, in order to test the visible set of objects. Another option would be to transform each object bounding box into post-projective space (world box * view * proj) and test against unit cube, but that would requires a additional matrix * matrix multiplication and 8 vector*matrix multiplication per object (what I would prefer sparing)...

Maybe a better way exists...

cheers,
Gregory

[Edited by - gjaegy on August 20, 2007 3:35:44 AM]

Share this post


Link to post
Share on other sites
gjaegy:
thanks!

> By the way, I still have 2 question:
> - in your demo you seem to only use a constant bias. Don't you need any slope > scale bias ?
z bias is key point of this method, they slightly different from usual approach. in second pass, z bias applied BEFORE final division, ie.
ZTest = (Z - Zbias) / W, where Zbias = bias / (ZMax - ZMin) (ZMax, ZMin from focus region basis). this bias mathematicaly, per-pixel correct (prove in 3.10 from paper) and should produce images without any surface artifacts (z acne, etc). so having only this correct bias, enough to produce artifact free images.

> - what are the role of EpsilonW, and Coef ? I didn't understand completely to > be honnest...
Coef - determine warping effect, large values should produce sagnificant warping(objects near viewer take large part of shadow map)
EpsilonW - works generally in complex scenes, when optimal warping go to singularity, and EpsilonW determine how far to singularity warping can be "pushed".

For culling you can:
1. transform final XPSM unit cube to world space with inverse transformation, and use this planes for culling
2. transfrom objects bounding objects into PPLS and test is it inside focus region box.

Share this post


Link to post
Share on other sites
I have tried transforming unit cube to world space using inverse matrix, then I construct the 6 frustum planes using for each of them 3 points on each face of the world space frustum.

This doesn't seems to work, I guess because of the discontinuity of the world space frustum. I guess it looks like this (if I understood correctly ;):



it seems objects get rejected because of near plane where they shouldn't...

If I transform object bounding boxes into XPSM unit cube then it works, however I have to transform objects bounding boxes (which I already have in world space) into post projection space (so I have additional matrix/vector transformations), which I would prefer getting rid of...

So, maybe I should have a look at the way I contruct my planes...

Share this post


Link to post
Share on other sites
actually it is quite strange. I have checked the front and near planes that result from the transformation of the unit cube into world space.

it simply seems that the plane normals are inverted, and I have no idea why.



the left red plane is the front plane, on the right the far plane. the normal should face each other in a normal case. the objects are between those planes.

I could simply check the orientation of the normals and fix them when required, however I would prefer understanding what's going on. any idea ?

Share this post


Link to post
Share on other sites
Vladislav, I have a last question: when i have a high Coef value (for instance 0.01), the objects are project outside the shadow map (i mean outside the [-1; 1] interval.

Do you have an idea, where the issue could come from?

See that screen shot:
screenshot

the shadowmap is actually containing 4 slices (CSM), the upper left corner of the shadow map contains the first slices. see how the rotor model is being rendered outside the shadowmap, hence the shadow not complete...

Thanks a lot...
Gregory

Share this post


Link to post
Share on other sites
is seems that bounding box not correctly constructed in PPLS. a) try to project casters/receivers in PPLS without W clipping, check how bounding volume points projected b) check how bounding box constructed, c) may be bounding box go extremely large\small.

Share this post


Link to post
Share on other sites
without w-clipping (or by setting epsilon to 0 if I understand correctly), i get the missing part being shadowed, however, i get some areas that get wrongly shadowed:

link

I clamp my shadow map sampling tex coords to be sure they stay in the right quarter, and each quarter has a border of 5 pixels (to be sure) with value 1.0.

I think I know what happens.

the shadow map is cleared with 0xFFFFFFF -> z=1.0f

but your algorithm makes it possible to have Z > 1.0. see this screen shot:

z-1.0

I have rendered (z - 1.0) (Z being the z component of the result of the Position*XPSMMAtrix)... not black areas means Zxpsm > 1.0f.

I am not sure it is possible to clear a render target with a number greater than 1.0 (as it takes a DWORD as argument)... maybe by blitting a full screen quad with a pixel shader setting the right value.

However, is that normal that I get Z values > 1.0 ??



[ADDED]

I solved the issue by simply clamping the z value of the pixel being drawn to [0.0, 1.0].

However, it seems the whole mesh is still not being shadowed correctly:

screen

coef = 0.6, epsilon = 0;

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this