Sign in to follow this  

Extracting the decimal

This topic is 4342 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Is there a function to extract only the decmial from an float value? I need to run it in a pixel shader, so it has to be relativily fast. Unfortunalty I have not needed this in the past, so I have no idea what it would be. I do not know the value passed in, so it can be 0.1 - 100000000.99 [Edited by - Valor Knight on January 26, 2006 1:11:26 AM]

Share this post


Link to post
Share on other sites

double number=35.049;
double decimal=number-(float)((int)number);
//decimal= 0.049


Would that work?

edit: just looked up the modf function as described above and it seems like a better solution.

Share this post


Link to post
Share on other sites
It needs to run in a pixel shader so #include <math.h> won't work, and I'm not sure the casts will work.

Quote:

frac - HLSL frac(x) Returns the fractional part f of x, such that f is a value greater than or equal to 0, and less than 1.


taken directly from the DirectX SDK, in the "HLSL Intrinsic Functions" part.

Share this post


Link to post
Share on other sites
^^ in fact, it seems I didn't understand the question either ^^
if % doesn't work with HLSL, then :

floor(frac(i * 0.1f) * 10.0f);

should retrieve the first decimal (I'm still not sure I did understand what your want : given 123.456, you want the function to return 3 ?)

Share this post


Link to post
Share on other sites
Quote:
Original post by paic
should retrieve the first decimal (I'm still not sure I did understand what your want : given 123.456, you want the function to return 3 ?)

no, I need it to return 0.456

all I needed to do was: decimal - floor(decimal) to get the decimal.
Thanks

[Edited by - Valor Knight on January 26, 2006 1:34:08 PM]

Share this post


Link to post
Share on other sites
Quote:
Original post by Valor Knight
Quote:
Original post by paic
should retrieve the first decimal (I'm still not sure I did understand what your want : given 123.456, you want the function to return 3 ?)

no, I need it to return 0.456

all I needed to do was: decimal - floor(decimal) to get the decimal.
Thanks


As paic suggested above, frac is the right thing to use.

BTW, the confusion was due to your terminology. "decimal" is a numbering system. You statement above makes as much sense as "all I needed to do was: binary - floor(binary) to get the binary"

Share this post


Link to post
Share on other sites

This topic is 4342 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this