Sign in to follow this  
dgreen02

Shadow map alpha testing question [SOLVED].

Recommended Posts

Hey guys...I've recently implemented realtime shadow maps into my game "Gang War", it's coming along very nicely, I'm just about done with 'em. I'm rendering 2 shadow maps every frame, a global shadow map that covers the entire city, and a local shadow map that is focused on the camera, and I blend between the two. Here are 2 videos showing how it looks in-game (you'll need DivX to watch these).... Anyways...I'm implementing alpha testing for fences, trees, etc. It doesn't seem to work. I was under the impression that shadow maps could easily support alpha testing in the shadows...ie: when rendering the scene from the light's perspective, just don't render texels that fail the alpha test. Though it doesn't seem to work. I've tried rendering the shadow map overlayed on screen and all parts that should fail the alpha test, and not be rendered, appear black (and cast a shadow). I'm explicitly enabling alpha testing right before rendering to the shadow maps, and setting the ALPHA_REF as well. Alpha testing works fine in every other part of the scene, though it doesn't seem to work for the shadow maps, here is another screenshot showing the problem....the shadow of the fence isn't alpha tested :-/ Is there something I'm missing? Is there a special way to use alpha testing when rendering shadow maps? I'd appreciate any help/advice you guys might have to offer. - Dan [Edited by - dgreen02 on December 10, 2005 3:44:03 AM]

Share this post


Link to post
Share on other sites
It should work, assuming you still set the texture containing the alpha, and setup your colorop/alphaop (remember a stage must have both a color and alphaop, or both be set to disable, in order to be valid) or pixel shader to fetch from the texture. I've done nVidia style shadow maps (ie: use depth buffer as a texture on ps.1.1 hardware) with alphatest and it worked as expected. Also, make sure you're not trying to over optimize your shadow geometry by stripping out UVs.

Share this post


Link to post
Share on other sites
Hmmm....Yup, I'm fairly sure I'm doing all that stuff correctly.

Here is another screenshot, this is the what the shadow map looks like when I sample the texture, instead of just outputting the depth in the shadow map pixel shader....and then display it...the shadow map is of D3DFMT_R32F format.

The circled areas are the problematic parts...I'm using alpha tested imposters for distant street lights, and the fence doesn't appear correct either :-/

Share this post


Link to post
Share on other sites
Anybody have any ideas? I'm still looking for a solution to this problem. Any suggestions would be appreciated!

The alpha testing happens after the pixel is sent from the shader, and before it's written to the surface, so the fact that I'm rendering to texture with a single channel shouldn't have anything to do with it, correct?

- Dan

Share this post


Link to post
Share on other sites
It's getting a bit late and my brain is starting to turn off... but I'm sure I've read about this before [oh]

Something to do with alpha testing still filling the depth buffer and only discarding the colour write.

Provided I am correct... If I can find a source for that or remember the details I'll get back to you [smile]

Jack

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Hi dgreen02,
your game look cool ! :)

Recent video cards does not support alpha testing with float rendertarget formats !
If your shadowmap is D3DFMT_R32F you must emulate alpha-testing manually
by killing pixels using texkill instruction.

Petr S.

Share this post


Link to post
Share on other sites
[EDIT: Oops, too slow!]
It is becuase alpha testing is not RT dependent, and most current GFX cards supporting FP render targets do not support post pixel shader blending operations (which includes alpha testing) when using them. I think there is either a cap to check for support or you can use CheckDeviceFormat or similar I think...

It's a real pain in the ass!

You can use texkill\clip() in your pixel shader though. One possibility is to render all opaque casters separately in a few large drawPrimitive calls (as you don't need their material info) and then render all alpha-testing casters with a pixel shader using texkill. This may get you some speed optimisations on some cards that take a different internal path if the pixel shader uses texkill.

Hope this helps,

Mike.

Share this post


Link to post
Share on other sites
I know, it is a bit off-topic (that is, not answering your question), but would you mind to explain your two shadow maps system a bit? I'm very curious! Thanks!

Share this post


Link to post
Share on other sites
That FP render targets thing sounds similar to what I was trying to remember earlier. The closest I could remember before reading that was that you had to emulate it using texkill/clip()...!

Quote:
Original post by roel
I know, it is a bit off-topic (that is, not answering your question), but would you mind to explain your two shadow maps system a bit? I'm very curious! Thanks!

Check out his developer journal - it's without a doubt one of the better ones. I forget when exactly, but he's talked about the various bits of technology used in his game/engine [smile]

cheers,
Jack

Share this post


Link to post
Share on other sites
Great...I appreciate the help guys! I'm sure I'll be able to get it working now.

roel - I'll also be posting some info in my developers journal, as well as some videos showing the all the new stuff, sometime today, so check it out if you're interested :-D

Thanks again for your help guys.

- Dan

Share this post


Link to post
Share on other sites
Heh...looks like I spoke too soon...I'm having some trouble getting this to work. I'm still very new to shaders, and I'm having trouble finding a good resource on this particular topic.


ps.2.0
//Constants
def c17, 0.20, 1.0, 0.0, 1.0
def c20, 0.0, 0.0, 0.0, 0.0

...

//Sample color map
texld r1, t2, s2

//Alpha test
mov r2, c20
sub r2.r, r1.a, c17.x
texkill r2

...

//Output pixel
mov oC0, r0


Alright this is what I have...it seems like it should work, but it fails to load.

If I take out either the sub, or the texkill it works fine (though no alpha testing). Could somebody explain to me why this doesn't work, I've been spinning my tires on this for a few hours now :-/ Mostly searching for info on how to properly use texkill.

When using texkill, I know that if any of the first 3 components are less than zero, the pixel will be killed. So I'm subtracting my "ALPHA_REF" value, stored in a constant, from the alpha value store in the texel I'm sampling...and storing it in the r component of a temp register...is there something not valid about that process?

Any help would be appreciated :-)

- Dan

Share this post


Link to post
Share on other sites
Yea true...good point about the psa and vsa utilities. I'll be making heavy use of these for all my shader development from now on.

These utilities sure are verbose...I could get spoiled with descriptive error messages like these [grin]. Now understanding it...that's another issue...

Quote:

scenerenderlocal.psh(142): error X5775: Dependent tex-op sequence too long (4th order).
A 1st order dependent tex-op is a tex[ld*|kill] instruction in which either: (1) an r# reg is
input (NOT t# reg), or (2) output r# reg was previously written, now being written AGAIN.
A 2nd order dependent tex-op occurs if: a tex-op reads OR WRITES to an r# reg whose contents,
BEFORE executing the tex-op, depend (perhaps indirectly) on the outcome of a 1st order dependent
tex-op. An (n)th order dependent tex-op derives from an (n-1)th order tex-op. A given tex-op may
be dependent to at most 3rd order (ps_2_0/x only).
assembly failed; no code produced


I take it that I'm abusing the r2 register with too many subsequent write commands?

If anybody could translate this into English, that would be great. In the meantime I'll be trying to figure it out.

- Dan

Share this post


Link to post
Share on other sites
Turns out the pixel shader code I wrote/posted was right...and I was being stupid somewhere else [grin]

Here's how it looks in the game now..thanks for your help guys!


Cheers!

- Dan

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this