Pixel shader executing wrong

Started by
2 comments, last by Plerion 11 years, 5 months ago
Hi everyone

Im coming here with a very strange problem. When im drawing rectangles i get strange artifacts along the adjacent edge of the two triangles im rendering.

As an example:
If im using a triangle fan with the following vertices:
0 1

3 2

I get artifacts at 0 and 2 as the edge of the two triangles is between 0 and 2, if im using a trianglelist with:
0 1 2
0 2 3

i get the same result (as expected).

If i make the triangle list the following:
0 1 3
1 2 3

i get artifacts at 1 and 3.

So what do i mean with artifacts? I have a little picture which shows them pretty well (its for triangle fan):
509458b04c4de7_fragment.png

The pink makes it more clear.

Now i was using PIX to debug my shader. There i found several very odd behaviours.

My shader was looking like that:
[source lang="cpp"]void PixelMain(in PixelShaderInput input, out float4 color : COLOR0)
{
float strokeX = strokeWidth.x / elementSize.x;
float strokeY = strokeWidth.x / elementSize.y;
float2 tex = input.texCoord;

if(tex.x <= strokeX || tex.y <= strokeY || (tex.x + strokeX) >= 1 || (tex.y + strokeY) >= 1)
color = tex2D(brushSampler, tex);
else
color = float4(1.0f, 0.35f, 0.7f, 1.0f);
}[/source]
In PIX i used single step debugging one of the two pixels with the artifacts and it showed me that my if-condition is met and it reads from the texture the correct value and then also executes the else-block writing the pink to the color. What the hell?

I thought it may be a source/asm mismatch and stepped through the bytecode which was like

if_ne r0.x, -r0.x
texld r1, v0, s0
else
mov r1, c2
endif


There again, it checks the if, it executes the texld and then executes also the mov r1, c2. Graphics card, what r u doin? Graphics card... staph!! biggrin.png

No matter to what i change my shader it also finds a way to bypass it so that for that pixel the pink is written. For example i did this:
[source lang="cpp"]void PixelMain(in PixelShaderInput input, out float4 color : COLOR0)
{
float strokeX = strokeWidth.x / elementSize.x;
float strokeY = strokeWidth.x / elementSize.y;
float2 tex = input.texCoord;
float2 pos = tex * elementSize.xy;
float2 posEnd = (float2(1, 1) - tex) * elementSize.xy;

color = float4(1.0f, 0.35f, 0.7f, 1);

if(tex.x <= strokeX)
color = tex2D(brushSampler, tex);
if(tex.y <= strokeY)
color = tex2D(brushSampler, tex);
if((tex.x + strokeX) >= 1)
color = tex2D(brushSampler, tex);
if((tex.y + strokeY) >= 1)
color = tex2D(brushSampler, tex);
}[/source]
I checked the values and saw that the first if must fail but the second is true while the rest is false. So far so good, first if failed, second if didnt, it executed the following byte code:
texld r1, v0, s0

Before the call r1 was: 1.0f, 0.35f, 0.7f, 1 (as expected)
After the call r1 was: 1.0f, 0.35f, 0.7f, 1 (NOT as expected, there is absolutely no pink in my brush!

What am i doing wrong? Is my graphics card fooling around with my? Why does this happen only at the edges where two triangles are adjacent?

/EDIT:
And to make things even more interesting: This error ONLY occurs on ATI cards, NVIDIA is all fine (except that not all 4 strokes have the same thickness).

Thanks in advance
Plerion
Advertisement
You didn't mention which shader model you are using, but I'm guessing you aren't familiar with branch and flatten attributes: http://msdn.microsof...610(VS.85).aspx . Try moving your tex2d call before the if and then just assign the correct float4 color inside the if-else.

In PIX i used single step debugging one of the two pixels with the artifacts and it showed me that my if-condition is met and it reads from the texture the correct value and then also executes the else-block writing the pink to the color. What the hell?


That's normal. Generally branches are implemented by executing both branches and lerping the results. This is because several pixels are processed in parallel, and they all need to run in lock-step and execute the same code.

You can explicitly specify dynamic branching by using the branch attribute like MikeBMcL said (and you can't use tex2D, ddx or ddy intrinsics inside the branch)

As for your artifact, it looks like you're using XNA or DX9? If so, are you taking into account the half texel offsetwhen sampling from your texture? Forgetting to do that is often the cause of artifacts along triangle edges.
Thanks you two, i had branching in mind but thought of it in a wrong way, but actually that wasnt the main cause of the artifacts. The problem was indeed the half texel offset and i changed my vertex shader to subtract a half pixel for my screen quads. Artifacts are gone.

This topic is closed to new replies.

Advertisement