Sign in to follow this  
Grafalgar

HLSL : Variables getting reset in pixel shader?

Recommended Posts

Hi all, I'm banging my head against this one, and have been searching for a solution for a days now without any success. A quick overview: I have an FX file that contains both a Vertex and Pixel shader. I precompile the FX file using the FX Compiler (fxc) and then use it through the D3D Effects Framework. Before I use the Effect, I set a few bool to check whether or not I'm using a texture and also whether lighting is enabled. This is so that I can bypass some texture sampling / lighting calculations if either of them are disabled for the particular material. In the Vertex Shader I do basic per-vertex lighting (after checking my LightingEnabled boolean), and then calculate the specular color. In the pixel shader I then use the specular color along with texture to determine the final pixel color. So far so good. Here's the tricky part. To compile the effect in Debug, I use the following command: fxc /Od /Zi /Tfx_2_0 /E$(InputName) /Fh "$(InputDir)$(InputName).h" "$(InputPath)" Od = Optimizations disabled Zi = Debugging info enabled In release I use the following: fxc /O3 /Tfx_2_0 /E$(InputName) /Fh "$(InputDir)$(InputName).h" "$(InputPath)" O3 = Optimization level 3 The problem is that when the /Od is specified, the two bools that I declare in the FX file are both set and valid during the Vertex Shader pass and Pixel Shader pass. When I enable optimizations (using /O3 or just omitting /Od) the bools are valid ONLY during the Vertex Shader pass, but are reset (false) during the Pixel Shader pass. I've verified this by brute force: Setting the vertex color during the Vertex Shader pass to Red (if my bool is true), and then setting the pixels to blue during the Shader Pass (also checking for the same boolean). My geometry comes out red. Done this a number of times and have verified time and again that the bool is true only during the Vertex Shader pass. But again, that's only when I do not disable optimizations. Here's some code: MyShader.fx:
struct VS_INPUT
{
    float4 	vPosition   	: POSITION;
    float3 	vNormal	    	: NORMAL0;
    float3 	vTexCoords  	: TEXCOORD0;
};

struct VS_OUTPUT
{
    float4 	vPosition	: POSITION;
    float3 	vTexCoords	: TEXCOORD0;
    float4	vColor		: COLOR0;
    float4      vSpec		: COLOR1;
};

texture  	DIFFUSE_TEXTURE;
bool	 	HAS_TEXTURE;
bool		ENABLE_LIGHTING;

float4 PS (VS_OUTPUT In) : COLOR
{
    float4 OutColor;
    
    if(HAS_TEXTURE)
    {
         // Sample the texture here.
         // If /Od is not specified, HAS_TEXTURE
         // is ALWAYS false
    }
    
    if(ENABLE_LIGHTING)
    {
         // Add the specular here (In.vSpec)
         // if /Od is not specified, ENABLE_LIGHTING
         // is ALWAYS false
    }

    return OutColor;
}

//================================================================

VS_OUTPUT VS(const VS_INPUT input)
{
	VS_OUTPUT Out = (VS_OUTPUT)0;
        Out.vPosition = input.vPosition;
	
	if(ENABLE_LIGHTING)
	{		
             // Lighting calculations go here.
             // ENABLE_LIGHTING and HAS_TEXTURE are
             // still valid here.
	}	
	
	return Out;
}

technique DefaultTechnique
{
    pass Pass0
    {
        AlphaBlendEnable = FALSE;
        AlphaTestEnable = FALSE;
	ShadeMode      = GOURAUD;
        
        MipFilter[0] = LINEAR;
     	MinFilter[0] = LINEAR;
       	MagFilter[0] = LINEAR;      
        
        ZWriteEnable = true;
        ZEnable = true;
        
        CullMode = CW;       

        // shaders
        VertexShader = compile vs_2_0 VS();
        PixelShader  = compile ps_2_0 PS();    
    }
}

The above is the trimmed down version of my full shader. I've omitted the many parts that are not relevant (vertex skinning, actual lighting calculations, etc). I'm seriously banging my head against. Without any code changes, specifying /Od makes everything work just fine. Leaving that out will never let my bools persist during the Pixel Shader pass. I have yet to find documentation in the DXSDK that would indicate what's going on, and my google searches have been fruitless. Any help would be GREATLY appreciated! Thanks in advance, Graf

Share this post


Link to post
Share on other sites
ps_2_0 doesn't support flow control. It's possible that the compiler does something to simulate that when you're compiling without optimisation, and if you want to check the differences, you can look at the shader assembly results of the two versions. However, even if this functionality is simulated, it will only cause more work for the pixel shader, not less, since no flow control is supported, so it will execute everything, then choose the result.

Share this post


Link to post
Share on other sites
I can't see anything obviously wrong with the code you've posted.

Which SDK version are you using? I'd prefer it to be the last resort, but it's not unheard of to get the compiler generate incorrect assembly..

Have you examined the assembly output? I forget the exact command line switches, but you can dump it to a colour coded HTML file. It may not be the easiest thing to read, but you can have a look to see what (if any) conditionals exist in the /O3 build - SM2 isn't very good at PS flow control so you'll probably find the compiler is pulling a few tricks to emulate them. It could be useful for you to explore this as your branches might not be saving you any execution time [wink]

Also, get friendly with a recent build of PIX for Windows - the 'pixel history' might be of interest. You can step through the results for a pixel and see why it received the final result you're seeing.

Finally, consider using uniform parameters and multiple effects. The actual VS/PS code remains much the same but you get a lot more technique's (hint: semantics can make this as programmatically accessible as your current system) and it allows the compiler the best chances at aggressive and correct optimization [grin]

hth
Jack

Share this post


Link to post
Share on other sites
As Jack said, you may wish to investigate using more techniques. I can't really explain the problem you're having, but here's how I've approached effects:

VS_OUTPUT VS(const VS_INPUT input, uniform bool bUseTexture)
{
if (bUseTexture)
{
// Do some tex coord stuff here, scrolling etc?
}
}

float4 PS(VS_OUTPUT In, uniform bool bUseTexture) : COLOR
{
if (bUseTexture)
{
// sample texture etc.
}
}



Then in your techniques:

technique untextured
{
pass p0
{
VertexShader = compile vs_2_0 VS(false);
PixelShader = compile ps_2_0 PS(false);
}
}

technique textured
{
pass p0
{
VertexShader = compile vs_2_0 VS(true);
PixelShader = compile ps_2_0 PS(true);
}
}



I use this technique to switch shaders between rigid, rigid skinned and smooth skinned (all of our stuff is textured, and if not then I use a "white" texture as untextured stuff is rarely used/intended).

I hope this helps, sorry I couldn't help with your actual problem though!

Share this post


Link to post
Share on other sites
Hi All,

The no-flow-control was right on the money. I read somewhere that what the pixel shader would do for flow-control is to evaluate both sides of the conditional, and then decide which result to use depending on the conditional.

So I tried the following instead:

OutColor = HAS_TEXTURE * tex2D(MyTextureSampler, In.vTexCoords) * In.vColor + (1 - HAS_TEXTURE) * In.vColor;
OutColor += (ENABLE_LIGHTING * In.vSpec);

And that seems to work just dandy =)

Thanks all!

Graf

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this