Hi, I'm trying to implement shadow mapping. I've gotten everything to work, but I'm having some issues making it look... up to snuff. What works is rendering the shadow map tot he back buffer, and copying it to a texture afterward. The problem, obviously, is that this means either my FoV for the light map is too small, or my final product is too pixelated. The obvious solutions would be to either use a smoothing algorithm to fade the edges of the shadows, or to increase the size of the original shadow map rendering. Attempt 1: Enlarge the original map. So I created a texture that's twice the size of my back buffer, and render to that. But, the map gets completely distorted and is unusable. This doesn't happen when the texture is less than or equal to the size of the back buffer. Is there a way around this? Attempt 2: Smooth the edges. The simplest method would be to sample a small kernel around the pixel in question and perform the depth test on each pixel, averaging the results. This works well, but to make a smooth fade I would need to sample a rather large kernel (probably atleast 5x5). But, it seems that the GPU isn't interested in doing that much math, let alone that many texture look ups. The only other thing I can think of off the top of my head is using 4 different depth maps, all with offsets.. but that doesn't seem like a great solution by any standards. Edit: I just thought of another good solution... if I scale the size of the shadow map based on the camera distance (this is a top down game) I can keep it looks good at all levels of detail. I'll still apply a Gaussian blur I think, though Edit2: This actually works pretty well, I'd still like to know if I can overcome the described issues though. I think I'll probably just use a third pass in order to apply the Gaussian blur, but for future reference... One major artifact still remains, as shown in this image... The bands are causing depth tests on the ground to fail incorrectly... anyone have any idea what may be causing these bands? [Edited by - Sharsnik on January 24, 2009 3:28:50 AM]

Quote:So I created a texture that's twice the size of my back buffer, and render to that. But, the map gets completely distorted and is unusable. This doesn't happen when the texture is less than or equal to the size of the back buffer.

Your depth buffer has to be at least the size of your rendertarget or bigger. So try to create a separate depth/stencil surface the size of your rendertarget and set it as depth buffer.

Quote:This works well, but to make a smooth fade I would need to sample a rather large kernel (probably atleast 5x5). But, it seems that the GPU isn't interested in doing that much math, let alone that many texture look ups.

What shadermodel are you using?

Quote:Original post by DraganOWhat shadermodel are you using?

vs_2_0 and ps_2_0.. here's the pixel shader for applying the map...

PS_OUTPUT pixelShaderLight( VS_MAP_OUTPUT IN ){	PS_OUTPUT OUT;		OUT.color = tex2D(s0, IN.texture0) * IN.texture0.z;	OUT.color += tex2D(s1, IN.texture1) * IN.texture1.z;	OUT.color += tex2D(s2, IN.texture2) * IN.texture2.z;	OUT.color += tex2D(s3, IN.texture3) * IN.texture3.z;	OUT.color += tex2D(s4, IN.texture4) * IN.texture4.z;	OUT.color += tex2D(s5, IN.texture5) * IN.texture5.z;		OUT.color.r *= IN.texture0.w;	OUT.color.g *= IN.texture0.w;	OUT.color.b *= IN.texture0.w;		float2 shadowUV;	float2 shadowUV2;		shadowUV.x = IN.lightSpace.x/IN.lightSpace.w/2 + 0.5;	shadowUV.y = -IN.lightSpace.y/IN.lightSpace.w/2 + 0.5;		float average = 0;			for (float x = -1; x <= 1; x++)	{		for (float y = -1; y <= 1; y++)		{			shadowUV2.x = shadowUV.x + x / 1024;			shadowUV2.y = shadowUV.y + y / 768;						if (tex2D(s7, shadowUV2).z < IN.lightSpace.z)			{				average += 1;			}						}	}		average = (1 - average / 9) * 0.6;//tex2D(s7, shadowUV).z;	OUT.color.r *= .4 + average;	OUT.color.g *= .4 + average;	OUT.color.b *= .4 + average;		return OUT;};

I do multitexturing and the map calculations in the same step.. and sense I'm going to need a decal pass and such eventually, I'm thinking that using 3 passes for the shadows may be a good idea.

Edit:

Quote:Original post by DraganOYour depth buffer has to be at least the size of your rendertarget or bigger. So try to create a separate depth/stencil surface the size of your rendertarget and set it as depth buffer.

Hmm... this fixes the warped map. But, the depth buffer doesn't seem to function properly on these calls. Objects just write directly to the screen without depth tests. Is there a render flag or something I need to enable to enable depth tests on the render target?

[Edited by - Sharsnik on January 22, 2009 3:25:27 AM]

Quote:Is there a render flag or something I need to enable to enable depth tests on the render target?

Try setting, D3DRS_ZENABLE and D3DRS_ZWRITEENABLE to true and D3DRS_ZFUNC to lessequal.

Quote:vs_2_0 and ps_2_0

Using shadermodel 3.0 will make it possible for you to use more instructions an texture lookups than using SM 2.0.

Quote:The bands are causing depth tests on the ground to fail incorrectly... anyone have any idea what may be causing these bands

Try using a floating point texture format like D3DFMT_R32F or D3DFMT_R16F for your rendertarget.

Cool, thanks everyone... this should look pretty sweet now :p

Okay... I'm starting to hate shaders...

Multi-texturing/shadow shader

PS_OUTPUT pixelShaderLight( VS_MAP_OUTPUT IN ){	PS_OUTPUT OUT;		OUT.color = tex2D(s0, IN.texture0) * IN.texture0.z;	OUT.color += tex2D(s1, IN.texture1) * IN.texture1.z;	OUT.color += tex2D(s2, IN.texture2) * IN.texture2.z;	OUT.color += tex2D(s3, IN.texture3) * IN.texture3.z;	OUT.color += tex2D(s4, IN.texture4) * IN.texture4.z;	OUT.color += tex2D(s5, IN.texture5) * IN.texture5.z;		OUT.color.rgb *= IN.texture0.w * 0.5f + 0.5f;		float2 shadowUV;		shadowUV.x = IN.lightSpace.y/IN.lightSpace.w/2 + 0.5;	shadowUV.y = -IN.lightSpace.y/IN.lightSpace.w/2 + 0.5;		float shadow = 1;		if (shadowUV.x > 0 && shadowUV.x < 1 && shadowUV.y > 0)	{				//shadow = lerp(1.5, .4, pow(abs(IN.texture0.w), .5));		//shadow = clamp(shadow, .65, 1);		//shadow *= shadow;		OUT.color.rgb *= .5;	}			OUT.color.rgb *= shadow;		return OUT;};

With this shader active (It's used to render terrain maps) I get 65 FPS in full screen 1440x900 mode (Shadow map is 2048x2048)

But if I add a fourth condition, such as....

if (shadowUV.x > 0 && shadowUV.x < 1 && shadowUV.y > 0)

to

if (shadowUV.x > 0 && shadowUV.x < 1 && shadowUV.y > 0 && shadowUV.y < 1)

My framerate drops to 40.

I'm thoroughly confused.

To work out why one version of the shader is slower Try using NVIDIA ShaderPerf or the AMD GPU Shader Analyzer.

Here's a few ways you might be able to optimize the shader. Try them out and see what's quickest.

- If you're sampling a texture with that shadowUV value, just set it to the border addressing mode and you can probably avoid the tests completely.

- Even if you weren't then sampling an appropriate 1x1 texture in that way might well be quicker than doing the tests.

- You could also try a [flatten] hint before that if statement in case the compiler decided to use a branch.

Well.. the clamping tests are rather moot, that was just to illustrate how adding one condition could destroy my FPS...

Here's the real shader... it runs at 41 FPS as well.

PS_OUTPUT pixelShaderLight( VS_MAP_OUTPUT IN ){	PS_OUTPUT OUT;		OUT.color = tex2D(s0, IN.texture0) * IN.texture0.z;	OUT.color += tex2D(s1, IN.texture1) * IN.texture1.z;	OUT.color += tex2D(s2, IN.texture2) * IN.texture2.z;	OUT.color += tex2D(s3, IN.texture3) * IN.texture3.z;	OUT.color += tex2D(s4, IN.texture4) * IN.texture4.z;	OUT.color += tex2D(s5, IN.texture5) * IN.texture5.z;		OUT.color.rgb *= IN.texture0.w * 0.5f + 0.5f;		float2 shadowUV;		shadowUV.x = IN.lightSpace.x/IN.lightSpace.w/2 + 0.5;	shadowUV.y = -IN.lightSpace.y/IN.lightSpace.w/2 + 0.5;		float shadow = 1;		if (IN.texture1.w - tex2D(s7, shadowUV).y > 0 && IN.lightSpace.z > tex2D(s7, shadowUV).x)	{				shadow = lerp(1.5, .4, pow(abs(IN.texture0.w), .5));		shadow = clamp(shadow, .65, 1);		shadow *= shadow;	}			OUT.color.rgb *= shadow;		return OUT;};

I'm not really sure how to use those shader tools well enough to be of any help.

I really just wanna know why it is that one more command can bring the frame rate from 65 to 40... is there something the shader does when it hits a certain number of operations?

Also, it seems to depend on the size of the map that's rendered. More there's a certain number of map vectors that cause the problem.