Jump to content
  • Advertisement
Sign in to follow this  

Variable precision libraries too precise

This topic is 4111 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I'm doing some work with a GPU simulator, and I'm trying to see what effects variable precision floating point operations will have. "Simple!" I say to myself, "I'll just toss in a variable-precision library into the math, limit it to however many bits I want, and compare the output to the full-precision sims." After some looking, I decided to give The GMP Library a try. However, unless I'm missing something, there's a minimum size enforced, for doubles, anyway. Following the instructions found here, I threw this together:
mpf_t A;
unsigned int uiPrec = mpf_get_prec(A);
if (uiPrec != 32)
	printf("Precision of A is %d\n", uiPrec);

It breaks, telling me the precision is actually 64. Changing the 32 to 128 works like a charm, though, which makes me think that I'm doing things correctly. So, a twofold query: 1) Has anyone had any experience with GMP? Am I, in fact, doing it right? 2) Does anyone know of any library that can do what I want (change the precision to lower than normal, where truncation and rounding will certainly come into play. My needs don't involve dealing with very big or very small numbers, just regular type.)? Thanks for the help! [Edit: Not to bump this back up to the top of anything, more to follow up for posterity: it's tricky to decide which bits to chop (the fraction bits will lose you some precision, and the exponent bits could be either precision (lsb, which wouldn't really make much sense) or range (msb). Chopping bits statically ("I want this number to have x exponent bits and y mantissa bits") means that hopefully the numbers won't go outside of the range expressible with x; if it does, it'll be infinity, so it'd be simpler just to clamp to infinity. Chopping bits dynamically ("I want a number with x bits, you decide how to split 'em up") is also a bit [heh] dicey. In the end, I couldn't find any code for CMUFloat, a library that seems to do exactly what I need, so I cobbled together my own. All it does right now is mask off mantissa bits, there's no fancy rounding or special case handling like the standard requires, but it sure does make my numbers (and pictures) ugly, so all's well!] [Edited by - jouley on October 2, 2007 2:27:26 PM]

Share this post

Link to post
Share on other sites
Sign in to follow this  

  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!