View more

View more

View more

### Image of the Day Submit

IOTD | Top Screenshots

### The latest, straight to your Inbox.

Subscribe to GameDev.net Direct to receive the latest updates and exclusive content.

# Problem hlsl code

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

7 replies to this topic

### #1lomateron  Members

Posted 18 October 2012 - 07:30 PM

[source lang="java"]this worksu = uint3(2,0, 0);r = Du.Load(u);this doesnt worksu = uint3(Cordd(2.0f), 0); //---Cordd(2.0f) return uint2(2,0)r = Du.Load(u);uint2 Cordd(float a){uint2 r;r.x=uint(a%3.0f);r.y=uint(a/3.0f);return r;}[/source]
In the complete method i return 'r' as a SV_Target, when i use the one that works the pixel is replaced by the values that 'r' has and when i use the one that doesnt the pixel mantains the same.

### #2Bacterius  Members

Posted 18 October 2012 - 07:44 PM

The modulo operator doesn't work as you think it does on floating-point numbers. See this link, down to the floating-point modulo section.

Basically, what it does is it divides the two operands and returns the fractional part (which is always between 0 inclusive and 1 inclusive). In your case, since you are converting that to an int after, you will always get zero no matter what. Using your example, you will get 2.0f % 3.0f = Frac(2.0f / 3.0f) = Frac(0.666..) = 0.666... Truncated down to an integer, you get zero. Not two.

You want to use fmod instead, probably. Or work in sufficiently large integers instead of floating-point numbers.

Edited by Bacterius, 18 October 2012 - 07:46 PM.

“If I understand the standard right it is legal and safe to do this but the resulting value could be anything.”

### #3lomateron  Members

Posted 18 October 2012 - 08:01 PM

thanks i have been 1 complete day in this problem.
Cord() method didnt returned me 0.666 and 'r' was getting the value that i expected in the end of the method but in the end the pixel wasnt getting it, so it was impossible to me to get that mistake.

### #4lomateron  Members

Posted 18 October 2012 - 08:25 PM

WAIT! its still not working, and the the module operator % is working like fmod() so it was the same thing,i am always sure that the float that enters the Cordd() method nevr has decimals. What else can cause this problem

Edited by lomateron, 18 October 2012 - 08:33 PM.

### #5Bacterius  Members

Posted 18 October 2012 - 08:31 PM

Can you try this:

[source lang="plain"]r.x = (uint(a) % 3) + frac(a);[/source]

“If I understand the standard right it is legal and safe to do this but the resulting value could be anything.”

### #6lomateron  Members

Posted 18 October 2012 - 08:39 PM

but r.x is an uint how can i add frac(a) to it.....
YESSS!!! it fianlly worked I used
(uint(a) % 3)
the thing was that 3 wasnt really 3 but another float variable so i used
(uint(a) % uint(f))
and it worked

Edited by lomateron, 18 October 2012 - 08:40 PM.

### #7Bacterius  Members

Posted 18 October 2012 - 08:44 PM

Oh yes, my bad. Note this won't work if your modulus is a fractional number like 2.5f or somesuch, but most of the time you are just trying to wrap a float around some integer. Though, note if you're only going to use the arguments as integers, you may as well pass them as integers.

Unsure why fmod didn't work, it should have. How did you use it?

“If I understand the standard right it is legal and safe to do this but the resulting value could be anything.”

### #8lomateron  Members

Posted 18 October 2012 - 09:13 PM

just
r.x=uint(fmod(a,f));
and it didnt work, using fmod() or %, Cordd() returned the correct value uint(2, 0)
I still dont know why in the end they dont work, the pixel was not change.

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.