I'm trying to work on a shader setup to blend lighting for a 2D game but have hit some unexpected problems that I hope someone here can shed some light on. The set up is I have 2 render targets with textures. On the first I render the games lights into the alpha channel. On the second I render the game map, player, shadows, etc I render a quad with using these targets as texture layer 0 & 1, applying a shader to mix them as needed. I've hit a problem before I even got to blending the textures in the shader, though. I'm forgetting the lighting layer for now, and just trying to get the quad to render the game without lighting, but have very weird results. Screenshot 1. Quad rendered without pixel shader using game texture - everything fine Screenshot 2. Quad rendered with pixel shader using game texture - shadows disappear! Code for the shader used:

sampler2D tex1 : register(s0);

float4 alphamask(float2 uv : TEXCOORD) : COLOR
{
	float4 Color;

	Color = tex2D( tex1, uv.xy );

	return Color;
}

The result of this shader confused me a lot because the shadows aren't drawn by masking with alpha - they're drawn with solid black (0xFF000000) quads & triangles in the middle of the map render as you can see in screenshot #1.. So the shader is sampling what you see in screenshot #1 - I don't get where it is sampling the correct texture for underneath the shadows from! A further test, I tried clamping the alpha output of the shader to 0.5: Screenshot 3. Quad rendered with pixel shader using game texture Code for the shader used:

sampler2D tex1 : register(s0);

float4 alphamask(float2 uv : TEXCOORD) : COLOR
{
	float4 Color;

	Color = tex2D( tex1, uv.xy );

	Color.a = 0.5;

	return Color;
}

Now things like the transparent box that surrounded the player appear! Surely when the player was rendered to the target texture, the transparent pixels were not written? Where is the shader finding these values? Can anybody tell me whats going on? [Edited by - mercior on June 22, 2009 1:03:12 PM]

Did you set the mip/min/mag filter states in the shader?

e.g.

sampler textureSampler = sampler_state
{
Texture = <Texture>;
MipFilter = LINEAR;
MinFilter = LINEAR;
MagFilter = LINEAR;
AddressU = Clamp;
AddressV = Clamp;
};

No, I have not. Infact, I've not seen that shader code before, what exactly does it do?

I'm compiling shaders into ps2.0 assembly using fxc, so the actual shader code I use looks more like:

ps_2_0dcl t0.xydcl_2d s0texld r0, t0, s0mov oC0, r0

I've briefly tried declaring my sampler in the .fx file as you suggested in the above post, but it doesn't appear to make any change to the assembly output.