Sign in to follow this  
lomateron

help with result of line of code

Recommended Posts

In HLSL 4.0 i have this line that is imposible for me to get the debugger to it(apart from having to wait around 15 minutes to build the solution in debug mode)

 

a=30030.0f;

b=11.0f;

c=true;

 

result = float(frac(a/b)==0.0f && c)

 

I expect  "result" to be 1.0f but I think "result" ends with value 0.0f, does anyone know why it ends with 0.0f, or is just paranoia? 

Share this post


Link to post
Share on other sites

I just realized i don't need to debug it using my whole application, I tested the line of code and "result" ends with a NaN, does someone knows why?

Edited by lomateron

Share this post


Link to post
Share on other sites


I tested the line of code and "result" ends with a NaN, does someone knows why?

 

Generally that would mean that b is zero.

 


result = float(frac(a/b)==0.0f && c)

 

An equality comparison against a special floating point value is always a red flag. Due to limited floating point precision, the result may not be exactly equal to 0.0f

Share this post


Link to post
Share on other sites

now I conclude it was just paranoia, the debug thing shows that the returned value is a NaN when rendering to a UNORM8, when rendering to a FLOAT32 it doesn't shows any value,the debug jumps all the code till the end. but I can see the color and new value of the pixel, and the results I expect are correct.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this