Sign in to follow this  
TaxCollector

Division and precision in c++

Recommended Posts

I've come upon a very annoying problem while doing division in c++.. I'll demonstrate the problem with an example: double a = double(500000000)/9; then the value of a = 55555555.555555552 but when I do: int b = 500000000; int c = 9; double a = double(b)/c; then the value of a = 55555556.000000000 Why is there a difference when I'm doing the exact same operation? For my game I want to be able to divide variables b and c without losing precision but I don't know how.

Share this post


Link to post
Share on other sites
Floating Point (especially the section "Accuracy problems")

double a = double(b)/c;

Here you divide a double with an integer, therefore the compiler decides for itself whichever of the two to convert back and forth.
In your case, it must have chosen to convert the result to an integer and then assign it to a double.

This should yield the value 55555555.555555552 (or something close to that in contrast to the FP semantics)

double a = double(b)/(double)c;

Share this post


Link to post
Share on other sites
There is no difference. The following program gives identical output for both values of a:
#include <iostream>

using namespace std;

int main()
{
double a = double(500000000)/9;
cout << fixed << a << endl;

int b = 500000000;
int c = 9;
a = double(b)/c;

cout << fixed << a << endl;
}

Output is:
55555555.555556
55555555.555556


Post your exact code and we might be able to see what's wrong.

Share this post


Link to post
Share on other sites
If I'm not mistaken, only one side of / is required to be a double to make the result double (the compiler shouldn't be free to choose the result type).

From the standard:
Quote:

--Otherwise, if either operand is double, the other shall be converted to double.


It is surprising to see that the OP's "wrong" result has been truncated "upwards".

Share this post


Link to post
Share on other sites
Yes it's true I've already tried converting both operands but I'm still getting the same result.

the exact code of my program is:

int b = 500000000;
int c = 9;
double a = double(b)/double(c);
a = double(500000000)/9;

And I check the value of variable 'a' in the debug mode of visual c++.

on line 3 , the value of a becomes 55555556.000000000
on line 4 , the value of a becomes 55555555.555555552

Does anyone have a clue?

even when I do:

double b = 500000000;
double c = 9;
double a = b/c;

the value of a is still 55555556.000000000

Share this post


Link to post
Share on other sites
That exact code gives equal output for both operations (55555555.555556) when compiled with g++. The discrepancy you see might be due to a debugger quirk. Try compiling and printing the result of the calculations through cout or printf.

By the way, what version of MSVC are you using? If it's a very old version (MSVC 6), there might be issues with nonstandard behaviour.

Share this post


Link to post
Share on other sites
Quote:
Original post by TaxCollector

And I check the value of variable 'a' in the debug mode of visual c++.

on line 3 , the value of a becomes 55555556.000000000
on line 4 , the value of a becomes 55555555.555555552


Can't replicate with MVS2008. All debug outputs show correct value.

What does debug mode mean anyway, as tooltip, as watch, compiled as debug configuration?

Share this post


Link to post
Share on other sites
After experimenting with the code I've found that at the very start of the program the calculation is done as expected.

It is only after calling the function

d3d->CreateDevice(D3DADAPTER_DEFAULT,
D3DDEVTYPE_HAL,
hWnd,
D3DCREATE_SOFTWARE_VERTEXPROCESSING,
&d3dpp,
&d3ddev);

(where d3ddev is an LPDIRECT3DDEVICE9;
and d3dpp is a D3DPRESENT_PARAMETERS;)

that the problem arises. So maybe this problem should've been posted in the DirectX part of the forum but I did not know the problem was caused by directX when I first posted the question. Because I'm a novice in DirectX, I do not know how to solve the issue.

Share this post


Link to post
Share on other sites
I believe DirectX modifies the way the floating point processor works. There is a flag "D3DCREATE_FPU_PRESERVE" that can be used (somewhere; I don't use DirectX [smile]) to keep the original behaviour. See if this is the source of your problem.

Share this post


Link to post
Share on other sites
Quote:
Original post by Funkymunky
what the hell?!
I think Microsoft needs to get off their freaking power trip and back the f off


I'm pretty sure they do it for a reason.

Quote:
From msdn
D3DCREATE_FPU_PRESERVE
Forces Direct3D to not change the floating-point unit (FPU) control word, running the pipeline using the precision of the calling thread. Without this flag, Direct3D defaults to setting the FPU to single-precision round-to-nearest mode. Using this flag with the FPU in double-precision mode will reduce Direct3D performance.

Warning: Portions of Direct3D assume FPU exceptions are masked. Unmasking such exceptions may lead to undefined behavior.


So using the flag can have a performance hit and can lead to undefined behaviour. The reason they change the FPU mode seems to be for performance reasons so is more of a 'smart move' rather than 'power trip.'

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this