Sign in to follow this  

Pixel shader executing wrong

This topic is 1901 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi everyone

Im coming here with a very strange problem. When im drawing rectangles i get strange artifacts along the adjacent edge of the two triangles im rendering.

As an example:
If im using a triangle fan with the following vertices:
0 1

3 2

I get artifacts at 0 and 2 as the edge of the two triangles is between 0 and 2, if im using a trianglelist with:
0 1 2
0 2 3

i get the same result (as expected).

If i make the triangle list the following:
0 1 3
1 2 3

i get artifacts at 1 and 3.

So what do i mean with artifacts? I have a little picture which shows them pretty well (its for triangle fan):
[img]http://imagr.eu/up/509458b04c4de7_fragment.png[/img]

The pink makes it more clear.

Now i was using PIX to debug my shader. There i found several very odd behaviours.

My shader was looking like that:
[source lang="cpp"]void PixelMain(in PixelShaderInput input, out float4 color : COLOR0)
{
float strokeX = strokeWidth.x / elementSize.x;
float strokeY = strokeWidth.x / elementSize.y;
float2 tex = input.texCoord;

if(tex.x <= strokeX || tex.y <= strokeY || (tex.x + strokeX) >= 1 || (tex.y + strokeY) >= 1)
color = tex2D(brushSampler, tex);
else
color = float4(1.0f, 0.35f, 0.7f, 1.0f);
}[/source]
In PIX i used single step debugging one of the two pixels with the artifacts and it showed me that my if-condition is met and it reads from the texture the correct value [b]and then also executes the else-block[/b] writing the pink to the color. What the hell?

I thought it may be a source/asm mismatch and stepped through the bytecode which was like
[code]
if_ne r0.x, -r0.x
texld r1, v0, s0
else
mov r1, c2
endif
[/code]

There again, it checks the if, it executes the texld and then executes also the mov r1, c2. Graphics card, what r u doin? Graphics card... staph!! [img]http://public.gamedev.net//public/style_emoticons/default/biggrin.png[/img]

No matter to what i change my shader it also finds a way to bypass it so that for that pixel the pink is written. For example i did this:
[source lang="cpp"]void PixelMain(in PixelShaderInput input, out float4 color : COLOR0)
{
float strokeX = strokeWidth.x / elementSize.x;
float strokeY = strokeWidth.x / elementSize.y;
float2 tex = input.texCoord;
float2 pos = tex * elementSize.xy;
float2 posEnd = (float2(1, 1) - tex) * elementSize.xy;

color = float4(1.0f, 0.35f, 0.7f, 1);

if(tex.x <= strokeX)
color = tex2D(brushSampler, tex);
if(tex.y <= strokeY)
color = tex2D(brushSampler, tex);
if((tex.x + strokeX) >= 1)
color = tex2D(brushSampler, tex);
if((tex.y + strokeY) >= 1)
color = tex2D(brushSampler, tex);
}[/source]
I checked the values and saw that the first if must fail but the second is true while the rest is false. So far so good, first if failed, second if didnt, it executed the following byte code:
texld r1, v0, s0

Before the call r1 was: 1.0f, 0.35f, 0.7f, 1 (as expected)
After the call r1 was: 1.0f, 0.35f, 0.7f, 1 (NOT as expected, there is absolutely no pink in my brush!

What am i doing wrong? Is my graphics card fooling around with my? Why does this happen only at the edges where two triangles are adjacent?

/EDIT:
[b]And to make things even more interesting: This error ONLY occurs on ATI cards, NVIDIA is all fine (except that not all 4 strokes have the same thickness).[/b]

Thanks in advance
Plerion Edited by Plerion

Share this post


Link to post
Share on other sites
You didn't mention which shader model you are using, but I'm guessing you aren't familiar with branch and flatten attributes: [url="http://msdn.microsoft.com/en-us/library/bb509610(VS.85).aspx"]http://msdn.microsof...610(VS.85).aspx[/url] . Try moving your tex2d call before the if and then just assign the correct float4 color inside the if-else.

Share this post


Link to post
Share on other sites
[quote name='Plerion' timestamp='1351899834' post='4996703']
In PIX i used single step debugging one of the two pixels with the artifacts and it showed me that my if-condition is met and it reads from the texture the correct value and then also executes the else-block writing the pink to the color. What the hell?
[/quote]

That's normal. Generally branches are implemented by executing both branches and lerping the results. This is because several pixels are processed in parallel, and they all need to run in lock-step and execute the same code.

You can explicitly specify dynamic branching by using the branch attribute like MikeBMcL said (and you can't use tex2D, ddx or ddy intrinsics inside the branch)

As for your artifact, it looks like you're using XNA or DX9? If so, are you taking into account the [url="http://drilian.com/2008/11/25/understanding-half-pixel-and-half-texel-offsets/"]half texel offset[/url]when sampling from your texture? Forgetting to do that is often the cause of artifacts along triangle edges. Edited by phil_t

Share this post


Link to post
Share on other sites
Thanks you two, i had branching in mind but thought of it in a wrong way, but actually that wasnt the main cause of the artifacts. The problem was indeed the half texel offset and i changed my vertex shader to subtract a half pixel for my screen quads. Artifacts are gone.

Share this post


Link to post
Share on other sites
Sign in to follow this