Sign in to follow this  
ProgrammerDX

HLSL being strange

Recommended Posts

Hello,

 

I am computing the dot product of two normalized vectors in HLSL, as far as I know, that should only return a float value in the range of [-1,1]. However, it's giving me a float value outside of that range, for some reason that I'm asking your help for finding out...

 

Code:

    float3 normal = normalize(float3(input.normal.x,input.normal.y,input.normal.z));
    float3 light = normalize(float3(0.25f, 0.5f, -1.0f));
    float m = abs(dot(normal, light));
    
    if( m > 2.0f )
        return float4(1,0,0,1);

This is in the pixel shader, I tested whether the dot value really gives back a float value above 1 by rendering a red pixel if it does... and it does.

 

Please help me find out what I am missing here...

 

I'm thinking it's somewhere in the value of input.normal... but then I'm wondering what it could be because I'm normalizing that vector anyway. It should give a unit vector regardless of weird values in input.normal

 

Thank you,

Edited by ProgrammerDX

Share this post


Link to post
Share on other sites

Bigger than 2 or (slightly) bigger than 1 ? The latter can happen due to numerical imprecision, but it should be in precision range of floats (something like 1.000001f). Some GPU hardware doesn't even have full IEEE compliance.

 

If you need the values to be in a desired range, clamp them.

Share this post


Link to post
Share on other sites

Ah yeah the input.normal has zero length. Strangely when I replace that with a float3(0,0,0) it doesn't bug quite the same way. Anyway, I've used a clamp() around dot() like unbird advised.

 

Thanks, closed.

Share this post


Link to post
Share on other sites

Ah yeah the input.normal has zero length. Strangely when I replace that with a float3(0,0,0) it doesn't bug quite the same way. Anyway, I've used a clamp() around dot() like unbird advised.

 

Thanks, closed.

 

Hint: If you need to clamp something in the range 0.0 - 1.0 have a look at the saturate intrinsic instead of clamp.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this