HLSL debugging: Damn those conditionals!

Started by
3 comments, last by EvilDecl81 18 years, 4 months ago
I have a pretty basic problem with debugging my shader. It works as such, I can make a breakpoint and step through the code - however, when I reach a conditional (if/else), it just walks past it, no matter what it would evaluate as. After it goes through the if-condition, it happily proceeds to ignore the else and goes through that code as well! But when the code runs without breakpoints, there is no problems, and it is obvious that the conditional works just fine. ARGH! Help me please, I can't see what I am doing wrong. This is how I set up the device:

DWORD behaviorFlags = D3DCREATE_SOFTWARE_VERTEXPROCESSING;

hr = m_pD3D->CreateDevice( m_d3dSettings.AdapterOrdinal(), D3DDEVTYPE_REF, m_hWndFocus, behaviorFlags, &m_d3dpp, &m_pd3dDevice );

And this is how I set up the shader (DEBUG_VS and DEBUG_PS are set):

	DWORD dwShaderFlags = 0;

	#ifdef DEBUG_VS
		dwShaderFlags |= D3DXSHADER_FORCE_VS_SOFTWARE_NOOPT;
	#endif
	#ifdef DEBUG_PS
		dwShaderFlags |= D3DXSHADER_FORCE_PS_SOFTWARE_NOOPT;
	#endif

dwShaderFlags |= D3DXSHADER_DEBUG;
dwShaderFlags |= D3DXSHADER_SKIPOPTIMIZATION;

    // If this fails, there should be debug output as to 
    // they the .fx file failed to compile
	if(FAILED(D3DXCreateEffectFromFile(GFXDevice::Instance()->GetDevice(), "ShadowMap.fx", NULL, NULL, dwShaderFlags, NULL, &g_pEffect, NULL)))
		return false;

The technique looks like this:

technique RenderScene
{
    pass p0
    {
        VertexShader = compile vs_1_1 VertScene();
        PixelShader = compile ps_2_0 PixScene();
    }
}

I assume that the VS and PS versions are good enough, since the conditionals do work when I don't set breakpoints...
Quote:CalvinI am only polite because I don't know enough foul languageQuote:Original post by superpigI think the reason your rating has dropped so much, Mercenarey, is that you come across as an arrogant asshole.
Advertisement
For 1.x shader targets, conditionals don't act as true flow control.

Usually what happens with an if/blah/else/blah in shader versions without true flow control is both the true and false results are computed then the relevent one selected based on the comparison. If you look at the disassembly for your shader you'll see this.

That's why you'll sometimes see the debugger step over both conditions of the if/else.

Simon O'Connor | Technical Director (Newcastle) Lockwood Publishing | LinkedIn | Personal site

Ok, that can explain the VS, but how about the PS? It is 2.0?
Quote:CalvinI am only polite because I don't know enough foul languageQuote:Original post by superpigI think the reason your rating has dropped so much, Mercenarey, is that you come across as an arrogant asshole.
While VS 2.0 has flow control instructions, PS 2.0 does not. However, PS 2.x and 3.0 do have flow control instructions.
John BoltonLocomotive Games (THQ)Current Project: Destroy All Humans (Wii). IN STORES NOW!
A couple of things:

1) You will get very strange results if you don't compile your shader with optimizations turned off. Code gets jumbled around quite a bit.

2) Most of the time you won't see flow control happen at the hardware level. You'll only see it in ps_3_0, yet most of the time it is impossible to use. Among the most common complaint I get is "Why doesn't my shader use an if statement?" And usually this is just because there was something like gradient (texture loads) inside the flow control.


EvilDecl81

This topic is closed to new replies.

Advertisement