Too many instructions in pixel shader

batchprogram · 2011-02-09T16:09:11

I've been working on a pixel shader that accepts up to 16 colors, 8 for "input" and 8 for "output". If a color in a sample matches the color in the input data, it swaps it out with the output color of the same index. I'm getting an error stating that my shader has too many instructions, and that shader model 2.0 only supports 64. I'm pretty sure this is because my shader isn't optimized, as every other solution I've tried won't compile. Shader: uniform extern texture Sample; sampler ScreenS = sampler_state { Texture = <Sample>; }; float4 input[8]; float4 output[8]; float4 Swap(float2 texCoord: TEXCOORD0) : COLOR { float4 color = tex2D(ScreenS, texCoord.xy); if (color.r == input[0].r && color.g == input[0].g && color.b == input[0]. color = output[0]; else if (color.r == input[1].r && color.g == input[1].g && color.b == input[1]. color = output[1]; else if (color.r == input[2].r && color.g == input[2].g && color.b == input[2]. color = output[2]; else if (color.r == input[3].r && color.g == input[3].g && color.b == input[3]. color = output[3]; else if (color.r == input[4].r && color.g == input[4].g && color.b == input[4]. color = output[4]; else if (color.r == input[5].r && color.g == input[5].g && color.b == input[5]. color = output[5];/* else if (color.r == input[6].r && color.g == input[6].g && color.b == input[6]. color = output[6]; else if (color.r == input[7].r && color.g == input[7].g && color.b == input[7]. color = output[7];*/ return color; } technique { pass P0 { PixelShader = compile ps_2_0 Swap(); } } If the comment is placed at the end of the else-if statement right above it's current position, it compiles fine, but it doesn't check the last 3 colors. How can this code be improved so that my criteria is met and uses <= 64 instructions?

Graphics and GPU Programming Programming

Started by batchprogram February 08, 2011 07:51 AM

10 comments, last by Sneftel 13 years, 2 months ago

Zoner

232

February 09, 2011 02:17 AM

Fully transparent pixels won't be visible, so to process them would be a big waste of processing time. Over half the pixels in the images I'm rendering are fully transparent.

Early outs in a shader almost never help with this problem. You should assume fully transparent pixels (via clip or alpha test) cost the same as all the others, because most of the time, they do.

http://www.gearboxsoftware.com/

Sneftel

1,788

February 09, 2011 04:09 PM

!all(color.rgb - input.rgb)

That totally doesn't do what you think it does. Perhaps you're looking for !any()?
[/quote]
The any function according to the hlsl reference docs: "This function is similar to the all HLSL intrinsic function. The any function determines if any components of the specified value are non-zero, while the all function determines if all components of the specified value are non-zero."[/quote]
Let's walk through this. Let's suppose color.rgb is (4,5,6) and input.rgb is (1,2,6). That is, one of the channels is the same, but not all of them.

So color.rgb - input.rgb is (3,3,0).

So all(color.rgb - input.rgb) is false.

So !all(color.rgb - input.rgb) is true.

See the issue yet?

!all means if they are all zero.
[/quote]
It does not. !all(a,b,c) is not the same as all(!a,!b,!c). However, !any(a,b,c) IS the same as all(!a,!b,!c).

For more info, read this.

Too many instructions in pixel shader

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Too many instructions in pixel shader

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines