• FEATURED

View more

View more

View more

### Image of the Day Submit

IOTD | Top Screenshots

### The latest, straight to your Inbox.

Subscribe to GameDev.net Direct to receive the latest updates and exclusive content.

# help with result of line of code

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

4 replies to this topic

### #1lomateron  Members

Posted 30 December 2013 - 12:43 AM

In HLSL 4.0 i have this line that is imposible for me to get the debugger to it(apart from having to wait around 15 minutes to build the solution in debug mode)

a=30030.0f;

b=11.0f;

c=true;

result = float(frac(a/b)==0.0f && c)

I expect  "result" to be 1.0f but I think "result" ends with value 0.0f, does anyone know why it ends with 0.0f, or is just paranoia?

### #2lomateron  Members

Posted 30 December 2013 - 01:01 AM

I just realized i don't need to debug it using my whole application, I tested the line of code and "result" ends with a NaN, does someone knows why?

Edited by lomateron, 30 December 2013 - 01:01 AM.

### #3phil_t  Members

Posted 30 December 2013 - 01:37 AM

I tested the line of code and "result" ends with a NaN, does someone knows why?

Generally that would mean that b is zero.

result = float(frac(a/b)==0.0f && c)

An equality comparison against a special floating point value is always a red flag. Due to limited floating point precision, the result may not be exactly equal to 0.0f

### #4lomateron  Members

Posted 30 December 2013 - 02:21 AM

Its NaN when :

a=30030.0f;

b=11.0f;

c=true;

Does someone know why?

### #5lomateron  Members

Posted 30 December 2013 - 03:14 AM

now I conclude it was just paranoia, the debug thing shows that the returned value is a NaN when rendering to a UNORM8, when rendering to a FLOAT32 it doesn't shows any value,the debug jumps all the code till the end. but I can see the color and new value of the pixel, and the results I expect are correct.

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.