# Shadow map alpha testing question [SOLVED].

## Recommended Posts

dgreen02    1428

##### Share on other sites
It should work, assuming you still set the texture containing the alpha, and setup your colorop/alphaop (remember a stage must have both a color and alphaop, or both be set to disable, in order to be valid) or pixel shader to fetch from the texture. I've done nVidia style shadow maps (ie: use depth buffer as a texture on ps.1.1 hardware) with alphatest and it worked as expected. Also, make sure you're not trying to over optimize your shadow geometry by stripping out UVs.

##### Share on other sites
dgreen02    1428
Hmmm....Yup, I'm fairly sure I'm doing all that stuff correctly.

Here is another screenshot, this is the what the shadow map looks like when I sample the texture, instead of just outputting the depth in the shadow map pixel shader....and then display it...the shadow map is of D3DFMT_R32F format.

The circled areas are the problematic parts...I'm using alpha tested imposters for distant street lights, and the fence doesn't appear correct either :-/

##### Share on other sites
dgreen02    1428
Anybody have any ideas? I'm still looking for a solution to this problem. Any suggestions would be appreciated!

The alpha testing happens after the pixel is sent from the shader, and before it's written to the surface, so the fact that I'm rendering to texture with a single channel shouldn't have anything to do with it, correct?

- Dan

##### Share on other sites
jollyjeffers    1570

Something to do with alpha testing still filling the depth buffer and only discarding the colour write.

Provided I am correct... If I can find a source for that or remember the details I'll get back to you [smile]

Jack

##### Share on other sites
Guest Anonymous Poster
Hi dgreen02,
your game look cool ! :)

Recent video cards does not support alpha testing with float rendertarget formats !
by killing pixels using texkill instruction.

Petr S.

##### Share on other sites
MikeWW    224
[EDIT: Oops, too slow!]
It is becuase alpha testing is not RT dependent, and most current GFX cards supporting FP render targets do not support post pixel shader blending operations (which includes alpha testing) when using them. I think there is either a cap to check for support or you can use CheckDeviceFormat or similar I think...

It's a real pain in the ass!

You can use texkill\clip() in your pixel shader though. One possibility is to render all opaque casters separately in a few large drawPrimitive calls (as you don't need their material info) and then render all alpha-testing casters with a pixel shader using texkill. This may get you some speed optimisations on some cards that take a different internal path if the pixel shader uses texkill.

Hope this helps,

Mike.

##### Share on other sites
roel    152
I know, it is a bit off-topic (that is, not answering your question), but would you mind to explain your two shadow maps system a bit? I'm very curious! Thanks!

##### Share on other sites
jollyjeffers    1570
That FP render targets thing sounds similar to what I was trying to remember earlier. The closest I could remember before reading that was that you had to emulate it using texkill/clip()...!

Quote:
 Original post by roelI know, it is a bit off-topic (that is, not answering your question), but would you mind to explain your two shadow maps system a bit? I'm very curious! Thanks!

Check out his developer journal - it's without a doubt one of the better ones. I forget when exactly, but he's talked about the various bits of technology used in his game/engine [smile]

cheers,
Jack

##### Share on other sites
dgreen02    1428
Great...I appreciate the help guys! I'm sure I'll be able to get it working now.

roel - I'll also be posting some info in my developers journal, as well as some videos showing the all the new stuff, sometime today, so check it out if you're interested :-D

Thanks again for your help guys.

- Dan

##### Share on other sites
roel    152
thanks to you too guys, i'll check it for sure!

##### Share on other sites
dgreen02    1428
Heh...looks like I spoke too soon...I'm having some trouble getting this to work. I'm still very new to shaders, and I'm having trouble finding a good resource on this particular topic.

ps.2.0//Constantsdef c17, 0.20, 1.0, 0.0, 1.0def c20, 0.0, 0.0, 0.0, 0.0...//Sample color maptexld r1, t2, s2//Alpha testmov r2, c20sub r2.r, r1.a, c17.xtexkill r2 ...//Output pixelmov oC0, r0

Alright this is what I have...it seems like it should work, but it fails to load.

If I take out either the sub, or the texkill it works fine (though no alpha testing). Could somebody explain to me why this doesn't work, I've been spinning my tires on this for a few hours now :-/ Mostly searching for info on how to properly use texkill.

When using texkill, I know that if any of the first 3 components are less than zero, the pixel will be killed. So I'm subtracting my "ALPHA_REF" value, stored in a constant, from the alpha value store in the texel I'm sampling...and storing it in the r component of a temp register...is there something not valid about that process?

Any help would be appreciated :-)

- Dan

##### Share on other sites
If by "failing to load" you mean that compilation fails, check the errors returned by the compilation function or using psa (DXSDK\Utilities\Bin\x86\psa).

##### Share on other sites
dgreen02    1428
Yea true...good point about the psa and vsa utilities. I'll be making heavy use of these for all my shader development from now on.

These utilities sure are verbose...I could get spoiled with descriptive error messages like these [grin]. Now understanding it...that's another issue...

Quote:
 scenerenderlocal.psh(142): error X5775: Dependent tex-op sequence too long (4th order). A 1st order dependent tex-op is a tex[ld*|kill] instruction in which either: (1) an r# reg is input (NOT t# reg), or (2) output r# reg was previously written, now being written AGAIN. A 2nd order dependent tex-op occurs if: a tex-op reads OR WRITES to an r# reg whose contents, BEFORE executing the tex-op, depend (perhaps indirectly) on the outcome of a 1st order dependent tex-op. An (n)th order dependent tex-op derives from an (n-1)th order tex-op. A given tex-op may be dependent to at most 3rd order (ps_2_0/x only).assembly failed; no code produced

I take it that I'm abusing the r2 register with too many subsequent write commands?

If anybody could translate this into English, that would be great. In the meantime I'll be trying to figure it out.

- Dan

##### Share on other sites
dgreen02    1428
Turns out the pixel shader code I wrote/posted was right...and I was being stupid somewhere else [grin]

Here's how it looks in the game now..thanks for your help guys!

Cheers!

- Dan