Hi everyone,

I'm trying to make a realistic shadow effect by flattening my object's geometry onto a plane (using D3DXMatrixShadow) and I've got a small problem:

My object is a tree which uses alpha testing to render partially transparent leaf textures, and I'm trying to use this alpha test on the shadows too. It works OK in that the shadows cast properly, but as you can see, there's no transparency (they are solid black). I am applying a translucent black shadow material to the flattened meshes before rendering them (which you can see works for the shadows of the small plants which don't use alpha testing on the shadows), but they still appear solid:

So, my question is - how can I use my translucent shadow material alongside the alpha tested textures which I have also applied to the shadows (in order to render the leaves properly)? It all works fine if I remove the textures (as for the small plant's shadows), but then I can't see my leaves properly.

Thanks!

What are the blend modes you're using? And your alpha func?

My alphafunc is D3DCMP_GREATEREQUAL and my blend modes are (D3DRS_SRCBLEND:D3DBLEND_SRCALPHA) and (D3DRS_DESTBLEND:D3DBLEND_INVSRCALPHA)

Hmmm that should work. It's been awhile since I used alpha test, but it should only be a test to discard a pixel. The only think I can think of is that your alpha really is less than your compare value. Or there is something else going on.

Have you tried bringing it up in Pix and making sure the states are what you think they should be?

I think it must be something to do with the texture not being blended correctly - some render state that I haven't set or something. When I don't apply the texture to the object, I get my shadow showing up as transparent, but of course there's no alpha testing so the leaves just show up as planes. Somehow the texture is refusing to take the material's alpha value.

Hmm. Well this is the time I would start doing my paranoid checks. Make sure that alpha blending is enabled, that your appropriate color write channels are enabled. Make sure your alpha ref isn't something absurd. Make sure none of your calls are failing.

Pix will help identify exactly what may be going wrong, and is extremely helpful for just this scenario. Have you used it before?

Hey Seabolt - I haven't used Pix - do you have a link? I wasn't sure where to find it on Google.

rukiruki - it turned out that the D3DTSS_ALPHAOP state needed to be set to D3DTOP_MODULATE as you suggested - it now works as I was hoping:

Thanks for your suggestions! This works a lot better than the crude shadows I was using before.

I'm glad your shadows are working now!

Pix is in your DirectX library. For example mine is at: C:\Program Files (x86)\Microsoft DirectX SDK (June 2010)\Utilities\bin\x86\PixWin.exe

From there you open it up, and give it a file path to your exe, and (optionally) a filepath to your assets.

MJP has a good tutorial for it here, It is infinitely helpful for debugging these things, just make you you enable debugging in your shaders.

Good luck, the game is looking good!