• Advertisement
Sign in to follow this  

Problem hlsl code

This topic is 1918 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

[source lang="java"]this works

u = uint3(2,0, 0);
r = Du.Load(u);

this doesnt works

u = uint3(Cordd(2.0f), 0); //---Cordd(2.0f) return uint2(2,0)
r = Du.Load(u);

uint2 Cordd(float a)
{
uint2 r;
r.x=uint(a%3.0f);
r.y=uint(a/3.0f);
return r;
}[/source]
In the complete method i return 'r' as a SV_Target, when i use the one that works the pixel is replaced by the values that 'r' has and when i use the one that doesnt the pixel mantains the same.

Share this post


Link to post
Share on other sites
Advertisement
The modulo operator doesn't work as you think it does on floating-point numbers. See [url="http://msdn.microsoft.com/en-us/library/windows/desktop/bb509631(v=vs.85).aspx#Additive_and_Multiplicative_Operators"]this link[/url], down to the floating-point modulo section.

Basically, what it does is it divides the two operands and returns the fractional part (which is always between 0 inclusive and 1 inclusive). In your case, since you are converting that to an int after, you will always get zero no matter what. Using your example, you will get 2.0f % 3.0f = Frac(2.0f / 3.0f) = Frac(0.666..) = 0.666... Truncated down to an integer, you get zero. Not two.

You want to use [url="http://msdn.microsoft.com/en-us/library/windows/desktop/bb509601(v=vs.85).aspx"]fmod[/url] instead, probably. Or work in sufficiently large integers instead of floating-point numbers. Edited by Bacterius

Share this post


Link to post
Share on other sites
thanks i have been 1 complete day in this problem.
Cord() method didnt returned me 0.666 and 'r' was getting the value that i expected in the end of the method but in the end the pixel wasnt getting it, so it was impossible to me to get that mistake.

Share this post


Link to post
Share on other sites
WAIT! its still not working, and the the module operator % is working like fmod() so it was the same thing,i am always sure that the float that enters the Cordd() method nevr has decimals. What else can cause this problem Edited by lomateron

Share this post


Link to post
Share on other sites
but r.x is an uint how can i add frac(a) to it.....
YESSS!!! it fianlly worked I used
(uint(a) % 3)
the thing was that 3 wasnt really 3 but another float variable so i used
(uint(a) % uint(f))
and it worked Edited by lomateron

Share this post


Link to post
Share on other sites
Oh yes, my bad. Note this won't work if your modulus is a fractional number like 2.5f or somesuch, but most of the time you are just trying to wrap a float around some integer. Though, note if you're only going to use the arguments as integers, you may as well pass them as integers.

Unsure why fmod didn't work, it should have. How did you use it?

Share this post


Link to post
Share on other sites
just
r.x=uint(fmod(a,f));
and it didnt work, using fmod() or %, Cordd() returned the correct value uint(2, 0)
I still dont know why in the end they dont work, the pixel was not change.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement