Jump to content
  • Advertisement
Sign in to follow this  
Valor Knight

Extracting the decimal

This topic is 4556 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Is there a function to extract only the decmial from an float value? I need to run it in a pixel shader, so it has to be relativily fast. Unfortunalty I have not needed this in the past, so I have no idea what it would be. I do not know the value passed in, so it can be 0.1 - 100000000.99 [Edited by - Valor Knight on January 26, 2006 1:11:26 AM]

Share this post


Link to post
Share on other sites
Advertisement

double number=35.049;
double decimal=number-(float)((int)number);
//decimal= 0.049


Would that work?

edit: just looked up the modf function as described above and it seems like a better solution.

Share this post


Link to post
Share on other sites
It needs to run in a pixel shader so #include <math.h> won't work, and I'm not sure the casts will work.

Quote:

frac - HLSL frac(x) Returns the fractional part f of x, such that f is a value greater than or equal to 0, and less than 1.


taken directly from the DirectX SDK, in the "HLSL Intrinsic Functions" part.

Share this post


Link to post
Share on other sites
Try

(((decimal * 10) % 10) / 10)

should give you the decimal only.
Imnot sure if % operator is available in shaders, but im sure that there is a function equivilant.

Share this post


Link to post
Share on other sites
^^ in fact, it seems I didn't understand the question either ^^
if % doesn't work with HLSL, then :

floor(frac(i * 0.1f) * 10.0f);

should retrieve the first decimal (I'm still not sure I did understand what your want : given 123.456, you want the function to return 3 ?)

Share this post


Link to post
Share on other sites
Quote:
Original post by paic
should retrieve the first decimal (I'm still not sure I did understand what your want : given 123.456, you want the function to return 3 ?)

no, I need it to return 0.456

all I needed to do was: decimal - floor(decimal) to get the decimal.
Thanks

[Edited by - Valor Knight on January 26, 2006 1:34:08 PM]

Share this post


Link to post
Share on other sites
Quote:
Original post by Valor Knight
Quote:
Original post by paic
should retrieve the first decimal (I'm still not sure I did understand what your want : given 123.456, you want the function to return 3 ?)

no, I need it to return 0.456

all I needed to do was: decimal - floor(decimal) to get the decimal.
Thanks


As paic suggested above, frac is the right thing to use.

BTW, the confusion was due to your terminology. "decimal" is a numbering system. You statement above makes as much sense as "all I needed to do was: binary - floor(binary) to get the binary"

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!