In the spirit of refactoring, I'm re-writing my lighting effect code from scratch. Naturally, there's unexplained bugs. This one, I don't know how to fix.

According to Bing, X3511 generally happens when someone inserts an instruction that forces the loop to unroll. But, without further ado, the relevant code:


struct LightData
{
int Type; // 0 = Directional, 1 = Point, 2 = Spot
float3 Direction;
float3 Position;
float Range;
float InnerCone;
float OuterCone;
};

struct VertexShaderOutput
{
float4 WorldPosition : POSITION0;
float4 UseWorldPosition : TEXCOORD0;
float2 UV : TEXCOORD1;
float4 Normal : NORMAL0;
};


LightData Lights[12];
int LightCount = 0;
int MaxLightCount = 12;


float4 PixelShaderA( VertexShaderOutput input ) : COLOR0
{
float4 colour = float4( 1, 1, 1, 1 );

for (int t = 0; t < LightCount && t < MaxLightCount; ++t)
{
float dotted = 1.0;
if (Lights[t].Type == 0)
{
float3 lightDir = normalize( Lights[t].Direction );

dotted = dot( input.Normal.xyz, lightDir );
if (dotted < 0) dotted = 0;
}
else if (Lights[t].Type == 1)
{
float3 toLight = Lights[t].Position - input.UseWorldPosition.xyz;

float3 lightDir = normalize( toLight );
float lightRange = length( toLight );

dotted = dot( input.Normal.xyz, lightDir );
dotted = dotted * (1.0 - ( lightRange / Lights[t].Range ));
if (dotted < 0) dotted = 0;
}
else if (Lights[t].Type == 2)
{
float3 lightDir = normalize( Lights[t].Direction );
float3 toLight = Lights[t].Position - input.UseWorldPosition.xyz;
toLight = 1; // If I comment this out, it glitches. <-------------------------------------------
float3 toLightDir = normalize( toLight );
float lightRange = length( toLight );

lightDir = 1; // If I comment this out, it glitches. <-------------------------------------------
float dotted2 = 1.0 - ( dot ( toLightDir, lightDir ) );
float aDotted2 = acos( dotted2 );
if ( dotted2 < 0 ) { dotted2 = 0; dotted = 0; }
else if ( aDotted2 <= Lights[t].OuterCone )
{
dotted = 0.0;
if (aDotted2 <= Lights[t].InnerCone)
dotted2 = ((aDotted2 / Lights[t].InnerCone) * (2.0 / 3.0)) + (1.0 / 3.0);
else
dotted2 =
(
(aDotted2 - Lights[t].InnerCone) /
(Lights[t].OuterCone - Lights[t].InnerCone)
) * (1.0 / 3.0);
dotted = dot( input.Normal.xyz, lightDir );
}
else dotted = 0;
}
colour *= dotted;
colour.a = 1.0;
}

return colour;
}


Which is rather odd, as both of those values are calculated in Directional and Point, not just Spot, and don't result in any glitches.

Help? Thanks.

ps_2_0 and ps_3_0 don't support indexing into constant registers at the assembly level. HLSL will let you do it, but when the shader is compiled all constant register accesses have to be explicit (static at compile-time). This means that it has to unroll your loop to support the code you wrote. If you're stuck with DX9 then your only alternative is to store your lighting parameters in a texture.

ps_2_0 and ps_3_0 don't support indexing into constant registers at the assembly level. HLSL will let you do it, but when the shader is compiled all constant register accesses have to be explicit (static at compile-time). This means that it has to unroll your loop to support the code you wrote. If you're stuck with DX9 then your only alternative is to store your lighting parameters in a texture.

The thing is, it's smaller than the loop in my previous lighting code, which compiled with no problems. That's what makes me think the actual error is something else (which I might not have adaquatly expressed). That, and just commenting out those lines shouldn't make the loop sufficiently smaller that it suddenly doesn't error.

I ran your pixel shader through GPU Shader Analyzer 1.57..., remembering to comment out your two check lines. I only had your reported error (X3511) with SM 2.x, it compiled ok with SM 3.0. The problem with SM 2.x is, there are only 32 float constant vectors. This function, unrolled, is apparently using 72 constant vecs (6 per light, you can easily squeeze this down to 3 per light). SM3.0 has 224 float constants, so you could get 50-60 lights with this function.

Thanks, but I've been compiling to SM 3.0 from the start. Could still be a FX Composer compiler glitch.

Whatever it was, it seems to have gone away with the current version of the lighting shader, around the time I was abstracting the lighting into a couple of functions and Vertex and Pixel lighting options (I only had Pixel lighting before - Accurate, but rather slow for lots of lights). Still be nice to know what it actually was, though. Aside from a Gremlin.

While we're on the topic of my LightData - I realized that it was putting a single LightData float into a vector constant just recently; I compacted it down to this:

struct LightData
{
int Type; // 0 = Directional, 1 = Point, 2 = Spot
float4 Ambient;
float4 Diffuse;
float3 Direction;
float3 Position;
float3 RangeInnerConeOuterCone;
// Light falloff variables.
float3 LinearQuadraticExponent;
};

Could it be reasonably compacted further?

Thanks.