Jump to content
  • Advertisement
Sign in to follow this  

Can anyone help with this shader issue? [solved]

This topic is 3542 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hey guys, i'm stuck. I've been banging my head against this bug for a while, and I could really use some help. I'm working on a 2d telestrator in 3d. My inputs are a series of points, and a normal to align the plane. Its for realtime tracking baseballs on air. I'm on rev 4 of trying to solve it. We had to give up on using FPP with alpha textures, they looked awful in the corners. The current rev looks like this, on ATi hardware: It is composed of two quads for the horizontal and vertical pieces. The corner is composed of 4 quads, so I can adjust the values for the softness on the inner/outer/.../... pieces. Here is what it looks like using solid materials: We are an nVidia shop. None of our clients use ATi. Here is what both GeForce and Quadro look like: The color's of the material's are comming from enum values being handed up in the first texcoord's x value. The code is straight forward:
#define	_3DUninited 0.0f // caused by parralel lines in stream
#define _3DMiter 1.0f
#define _3DNoMitre 2.0f
#define Planar 5.0f // keep planar values higher than this
#define PlanarLeftTurn 6.0f
#define PlanarRightTurn	7.0f
#define PlanarCorner 8.0f
#define False 0.0f

#define UpperRightIsOuter 0
#define UpperRightIsInner 1
#define UpperLeftIsOuter 2
#define UpperLeftIsInner 3
#define LowerRightIsOuter 4
#define LowerRightIsInner 5
#define LowerLeftIsOuter 6
#define LowerLeftIsInner 7

#define Inner 0.0f
#define OuterD1 1.0f
#define Outer 2.0f
#define OuterD2 3.0f
#define FirstPiece   1.0f // eg, use D1
#define SecondPiece  3.0f // eg, use D2

// vertex shaders must accpet our vertex format, this its definition 
	float4 Position:POSITION0;
	float3 Normal:NORMAL0;
	float3 Binormal:BINORMAL0;
	float3 Tangent:TANGENT0;
	float2 UV:TEXCOORD0;
	float2 UV1:TEXCOORD1;
	float2 UV2:TEXCOORD2;

struct linedraw_VS_OUTPUT {
    float4 Position	        : POSITION;
    float4 Normal		: NORMAL;
    float4 D1			: TEXCOORD0;
    float4 D2		       : TEXCOORD1;
    float4 modelVertPos   : TEXCOORD2;
    float4 FaceCenter	       : TEXCOORD3;
    int Type                      : TEXCOORD4;
    float4 LitMaterial	       : COLOR0; 						         

linedraw_VS_OUTPUT linedrawVS(VS_INPUT IN)
	linedraw_VS_OUTPUT OUT = (linedraw_VS_OUTPUT)0;
	OUT.Position = mul( float4(IN.Position.xyz , 1.0) , worldViewProj);
	OUT.Type = (int)IN.UV.x;
	return OUT;	

// standard pixel shader
float4 linedrawPS(linedraw_VS_OUTPUT IN) : COLOR
	float4 result = float4(0, 0, 0, 0);

	int myType = IN.Type;
	if (myType == (int)Inner)
		result = float4(1, 0, 0, 1);
	if (myType == (int)Outer)
		result = float4(0, 1, 0, 1);
	if (myType == (int)SecondPiece)
		result = float4(1, 1, 1, 1);
	if (myType == (int)FirstPiece)
		result = float4(1, 0, 1, 1);
	return result;

#define FXComposer

technique technique_main
    pass p0 
#ifdef FXComposer    
    	AlphaBlendEnable = True;
		SrcBlend = SrcAlpha;
		DestBlend = InvSrcAlpha;
		VertexShader = compile vs_3_0 linedrawVS();
		PixelShader  = compile ps_3_0 linedrawPS();
Tracing into PIX is giving odd results. The final debug view is telling me that I am getting junk output (0, 0, 0, 0) The issue is that the actual pixel debugger say's i'm getting correct output: (white in this case) Here is the PIXRun if anyone wants to see: http://ajtsystems.com/mcollins/shaderoddness/NvidiaBad.PIXRun I've verified that the value's coming out of the vertex shader are constant for each triangle, so it doesn't seem like the interpolator could be stomping anything. I've tried handing in the data as float's, int's, etc. It seems like the nVidia hardware can't handle doing a branch on a value that comes out of a vertex shader's interpolator. It looks like its sending random stream processors off with bad data. [a guess] Its odd, because as you rotate the mesh, the bad areas jump around. We branch on uniform's all the time. Handing up uniforms to the pixel shader isn't feasible this time--the pipeline stall's too badly for all the constant changes. Right now, the only other solution I have is up upload coefficients for all the values, and just zero stuff out; however, I'm already at the limit of our current vertex format. It would be difficult to extent it again [not to mention that much larger]. Has anyone see anything like this? Thanks in advance. On a prior rev, I tried doing this with vertex texture fetches, and saw a similar inability to draw uniformly. [Edited by - Mathucub on December 4, 2008 7:54:54 AM]

Share this post

Link to post
Share on other sites
Regardless of the casting to ints, I believe shaders always use floats, at least in shader model 3.

Have you tried adding some tolerance to those calculations. Something like:

if (abs(myType - Inner) < 0.01f)
result = float4(1, 0, 0, 1);

Share this post

Link to post
Share on other sites
looks like the abs trick worked.
I tried that yesterday, but on our c++ code our epsilon is 0.001.
To work in the shader I have to crank it up to 0.1

That seems like a lot of noise, but it works!

Share this post

Link to post
Share on other sites
Sign in to follow this  

  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!