What is wrong with my compute shader?

Started by
0 comments, last by Chetanhl 9 years, 2 months ago

Hello! I am trying to write a compute shader which rasterizes a single 2D triangle. Here is the code for the shader:


cbuffer rasterizer_params : register(b0)
{
    float3 default_color, tri_color;
    int num_tris;
    uint output_width, output_height;
	float3 padding;
}

StructuredBuffer<int2> input_vertices : register(b0);
RWTexture2D<float4> output_texture : register(u0);

float3 barycentric(int2 pos, int2 a, int2 b, int2 c)
{
    float3 res; 

    float2 v0 =  pos - a;
    float2 v1 = b - a;
    float2 v2 = c - a;
    
    float d20 = dot(v2, v0);
    float d12 = dot(v1, v2);
    float d22 = dot(v2, v2);
    float d10 = dot(v1, v0);
    float d11 = dot(v1, v1);
    float d21 = dot(v2, v1);

    float denom = d22*d11 - d21*d12;

    res.y = (d10*d22 - d20*d21) / denom;
    res.z = (d20*d11 - d10*d12) / denom;
    res.x = 1.0f - (res.y + res.z);
    return res;
}

float3 rasterize(int2 pos, int2 vert0, int2 vert1, int2 vert2)
{
    float3 res = barycentric(pos, vert0, vert1, vert2);
    
    if(res.x >= 0.0f && res.y >= 0.0f && res.z >= 0.0f)
        return tri_color;
    else
        return default_color;
}

[numthreads(32, 32, 1)]
void CSMain(uint2 dispatch_tid : SV_DispatchThreadID)
{
	float3 pix_color;

	pix_color = rasterize(
		int2(dispatch_tid.x, dispatch_tid.y),
		int2(0, 0),
		int2(25, 0),
		int2(0, 25));


	output_texture[dispatch_tid.xy] = float4(pix_color.x, pix_color.y, pix_color.z, 1.0f);
}

The output is a completely black texture (meaning that none of the pixels are passing the rasterization test). I've tried running through my code in the graphics debugger. However, I've noticed that I can't read the values of a lot of variables in the code (or the appear as NaN). I assume that this is due to the way that the shader is compiled but it makes the debugger almost useless if I can't examine the values of certain variables over the execution of my program. What gives?

J.W.
Advertisement

Hi, for debugging your shaders you can use these compiler flags - D3DCOMPILE_DEBUG | D3DCOMPILE_SKIP_OPTIMIZATION | D3DCOMPILE_PREFER_FLOW_CONTROL.

More info here.

Also, I have noticed many times in graphics debugger that variables that are no longer needed are assigned NAN.

You might want to try RenderDoc - for debugging. It's very easy to use and similar to pix & visual graphics debugger.

I find it much better than visual graphics debugger.

For example - One benefit is that it will show you the values in the registers which are retained unless overwritten by some other operation. Also you can go both backwards & forward while debugging.

My Game Development Blog : chetanjags.wordpress.com

This topic is closed to new replies.

Advertisement