Little known problem with Direct3D - Floating point precision

Started by
0 comments, last by jollyjeffers 18 years, 7 months ago
Just something we encountered at work that I thougth was worth sharing. Quote from http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_c/directx/technicalarticles/topissues.asp
Manipulating the Float-Point Control Word As a debugging aid, some developers have been enabling exceptions on the floating-point unit (FPU) via manipulations of the floating-point control word. Doing this is highly problematic and will likely cause the process to crash. Just like the calling convention requires the ebx register be preserved, the majority of the system assumes that the FPU is in a default state, will give reasonable results, and won't generate exceptions. Drivers and other system components will often compute results assuming that standard error values will appear in the registers for bad conditions, and if exceptions are enabled, these will go unhandled and result in crashes. Direct3D will set the floating-point unit to "single-precision, round-to-nearest" as part of initialization, unless the D3DCREATE_FPU_PRESERVE flag is used, in which case the FP control word is untouched. In any libraries where we need to have different rounding rules or other behavior�such as dealing with software vertex shaders or compilation�we save and restore the control word. If the game needs to make use of non-standard rounding or FPU exceptions, it should save and restore the FP control word and be sure to not call any external code not proven to be safe from these problems, including system APIs.
The short of it is that if you do any computation that relies on using doubles for more accurate artihmetic, you had better specify D3DCREATE_FPU_PRESERVE if you are using Direct3D for your visualisation. Either that or you had best set the floating point control word back into a mode that is more suitable for you. We found that our CSG and self intersection tests were failing because the necessary level of precision just wasn't there any more. - Phil [Edited by - Phillip Martin on September 12, 2005 6:27:23 AM]
Advertisement
Good post [smile]

It's never bothered me personally, but I have read about this sort of thing causing people problems before - some of them really obscure to track down.

Although, to my knowledge, most multimedia/games scenarios won't really need the extra precision enough to make it an issue - which is good!

Cheers,
Jack

<hr align="left" width="25%" />
Jack Hoxley <small>[</small><small> Forum FAQ | Revised FAQ | MVP Profile | Developer Journal ]</small>

This topic is closed to new replies.

Advertisement