Hello,

I am computing the dot product of two normalized vectors in HLSL, as far as I know, that should only return a float value in the range of [-1,1]. However, it's giving me a float value outside of that range, for some reason that I'm asking your help for finding out...

Code:

    float3 normal = normalize(float3(input.normal.x,input.normal.y,input.normal.z));
    float3 light = normalize(float3(0.25f, 0.5f, -1.0f));
    float m = abs(dot(normal, light));
    
    if( m > 2.0f )
        return float4(1,0,0,1);

This is in the pixel shader, I tested whether the dot value really gives back a float value above 1 by rendering a red pixel if it does... and it does.

Please help me find out what I am missing here...

I'm thinking it's somewhere in the value of input.normal... but then I'm wondering what it could be because I'm normalizing that vector anyway. It should give a unit vector regardless of weird values in input.normal

Thank you,

Bigger than 2 or (slightly) bigger than 1 ? The latter can happen due to numerical imprecision, but it should be in precision range of floats (something like 1.000001f). Some GPU hardware doesn't even have full IEEE compliance.

If you need the values to be in a desired range, clamp them.

Ah yeah the input.normal has zero length. Strangely when I replace that with a float3(0,0,0) it doesn't bug quite the same way. Anyway, I've used a clamp() around dot() like unbird advised.

Thanks, closed.