Sign in to follow this  

Can anyone help with this shader issue? [solved]

This topic is 3296 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hey guys, i'm stuck. I've been banging my head against this bug for a while, and I could really use some help. I'm working on a 2d telestrator in 3d. My inputs are a series of points, and a normal to align the plane. Its for realtime tracking baseballs on air. I'm on rev 4 of trying to solve it. We had to give up on using FPP with alpha textures, they looked awful in the corners. The current rev looks like this, on ATi hardware: It is composed of two quads for the horizontal and vertical pieces. The corner is composed of 4 quads, so I can adjust the values for the softness on the inner/outer/.../... pieces. Here is what it looks like using solid materials: We are an nVidia shop. None of our clients use ATi. Here is what both GeForce and Quadro look like: The color's of the material's are comming from enum values being handed up in the first texcoord's x value. The code is straight forward:
#define	_3DUninited 0.0f // caused by parralel lines in stream
#define _3DMiter 1.0f
#define _3DNoMitre 2.0f
#define Planar 5.0f // keep planar values higher than this
#define PlanarLeftTurn 6.0f
#define PlanarRightTurn	7.0f
#define PlanarCorner 8.0f
#define False 0.0f

#define UpperRightIsOuter 0
#define UpperRightIsInner 1
#define UpperLeftIsOuter 2
#define UpperLeftIsInner 3
#define LowerRightIsOuter 4
#define LowerRightIsInner 5
#define LowerLeftIsOuter 6
#define LowerLeftIsInner 7

#define Inner 0.0f
#define OuterD1 1.0f
#define Outer 2.0f
#define OuterD2 3.0f
#define FirstPiece   1.0f // eg, use D1
#define SecondPiece  3.0f // eg, use D2

// vertex shaders must accpet our vertex format, this its definition 
struct SHADER_NO_VTABLE VS_INPUT
{
	float4 Position:POSITION0;
	float3 Normal:NORMAL0;
	float3 Binormal:BINORMAL0;
	float3 Tangent:TANGENT0;
	float2 UV:TEXCOORD0;
	float2 UV1:TEXCOORD1;
	float2 UV2:TEXCOORD2;
};

struct linedraw_VS_OUTPUT {
    float4 Position	        : POSITION;
    float4 Normal		: NORMAL;
    float4 D1			: TEXCOORD0;
    float4 D2		       : TEXCOORD1;
    float4 modelVertPos   : TEXCOORD2;
    float4 FaceCenter	       : TEXCOORD3;
    int Type                      : TEXCOORD4;
    float4 LitMaterial	       : COLOR0; 						         
};

linedraw_VS_OUTPUT linedrawVS(VS_INPUT IN)
{
	linedraw_VS_OUTPUT OUT = (linedraw_VS_OUTPUT)0;
	OUT.Position = mul( float4(IN.Position.xyz , 1.0) , worldViewProj);
	
	OUT.Type = (int)IN.UV.x;
	 
	return OUT;	
}

// standard pixel shader
float4 linedrawPS(linedraw_VS_OUTPUT IN) : COLOR
{   
	float4 result = float4(0, 0, 0, 0);

	int myType = IN.Type;
	
	if (myType == (int)Inner)
	{
		result = float4(1, 0, 0, 1);
	}
	
	if (myType == (int)Outer)
	{
		result = float4(0, 1, 0, 1);
	}
	
	if (myType == (int)SecondPiece)
	{
		result = float4(1, 1, 1, 1);
	}
	
	if (myType == (int)FirstPiece)
	{
		result = float4(1, 0, 1, 1);
	}
	
	return result;
}


#define FXComposer

//-----------------------------------
technique technique_main
{
    pass p0 
    {		
#ifdef FXComposer    
    	AlphaBlendEnable = True;
		SrcBlend = SrcAlpha;
		DestBlend = InvSrcAlpha;
#endif	
		VertexShader = compile vs_3_0 linedrawVS();
		PixelShader  = compile ps_3_0 linedrawPS();
    }
}
Tracing into PIX is giving odd results. The final debug view is telling me that I am getting junk output (0, 0, 0, 0) The issue is that the actual pixel debugger say's i'm getting correct output: (white in this case) Here is the PIXRun if anyone wants to see: http://ajtsystems.com/mcollins/shaderoddness/NvidiaBad.PIXRun I've verified that the value's coming out of the vertex shader are constant for each triangle, so it doesn't seem like the interpolator could be stomping anything. I've tried handing in the data as float's, int's, etc. It seems like the nVidia hardware can't handle doing a branch on a value that comes out of a vertex shader's interpolator. It looks like its sending random stream processors off with bad data. [a guess] Its odd, because as you rotate the mesh, the bad areas jump around. We branch on uniform's all the time. Handing up uniforms to the pixel shader isn't feasible this time--the pipeline stall's too badly for all the constant changes. Right now, the only other solution I have is up upload coefficients for all the values, and just zero stuff out; however, I'm already at the limit of our current vertex format. It would be difficult to extent it again [not to mention that much larger]. Has anyone see anything like this? Thanks in advance. On a prior rev, I tried doing this with vertex texture fetches, and saw a similar inability to draw uniformly. [Edited by - Mathucub on December 4, 2008 7:54:54 AM]

Share this post


Link to post
Share on other sites
Regardless of the casting to ints, I believe shaders always use floats, at least in shader model 3.

Have you tried adding some tolerance to those calculations. Something like:

if (abs(myType - Inner) < 0.01f)
{
result = float4(1, 0, 0, 1);
}

Share this post


Link to post
Share on other sites
looks like the abs trick worked.
I tried that yesterday, but on our c++ code our epsilon is 0.001.
To work in the shader I have to crank it up to 0.1

That seems like a lot of noise, but it works!
Thanks!

Share this post


Link to post
Share on other sites

This topic is 3296 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this