Jump to content
  • Advertisement
Sign in to follow this  
Phillip Martin

Little known problem with Direct3D - Floating point precision

This topic is 4815 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Just something we encountered at work that I thougth was worth sharing. Quote from http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_c/directx/technicalarticles/topissues.asp
Manipulating the Float-Point Control Word As a debugging aid, some developers have been enabling exceptions on the floating-point unit (FPU) via manipulations of the floating-point control word. Doing this is highly problematic and will likely cause the process to crash. Just like the calling convention requires the ebx register be preserved, the majority of the system assumes that the FPU is in a default state, will give reasonable results, and won't generate exceptions. Drivers and other system components will often compute results assuming that standard error values will appear in the registers for bad conditions, and if exceptions are enabled, these will go unhandled and result in crashes. Direct3D will set the floating-point unit to "single-precision, round-to-nearest" as part of initialization, unless the D3DCREATE_FPU_PRESERVE flag is used, in which case the FP control word is untouched. In any libraries where we need to have different rounding rules or other behavior�such as dealing with software vertex shaders or compilation�we save and restore the control word. If the game needs to make use of non-standard rounding or FPU exceptions, it should save and restore the FP control word and be sure to not call any external code not proven to be safe from these problems, including system APIs.
The short of it is that if you do any computation that relies on using doubles for more accurate artihmetic, you had better specify D3DCREATE_FPU_PRESERVE if you are using Direct3D for your visualisation. Either that or you had best set the floating point control word back into a mode that is more suitable for you. We found that our CSG and self intersection tests were failing because the necessary level of precision just wasn't there any more. - Phil [Edited by - Phillip Martin on September 12, 2005 6:27:23 AM]

Share this post


Link to post
Share on other sites
Advertisement
Good post [smile]

It's never bothered me personally, but I have read about this sort of thing causing people problems before - some of them really obscure to track down.

Although, to my knowledge, most multimedia/games scenarios won't really need the extra precision enough to make it an issue - which is good!

Cheers,
Jack

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!