Division and precision in c++

This topic is 3697 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

Recommended Posts

I've come upon a very annoying problem while doing division in c++.. I'll demonstrate the problem with an example: double a = double(500000000)/9; then the value of a = 55555555.555555552 but when I do: int b = 500000000; int c = 9; double a = double(b)/c; then the value of a = 55555556.000000000 Why is there a difference when I'm doing the exact same operation? For my game I want to be able to divide variables b and c without losing precision but I don't know how.

Share on other sites
Floating Point (especially the section "Accuracy problems")

double a = double(b)/c;

Here you divide a double with an integer, therefore the compiler decides for itself whichever of the two to convert back and forth.
In your case, it must have chosen to convert the result to an integer and then assign it to a double.

This should yield the value 55555555.555555552 (or something close to that in contrast to the FP semantics)

double a = double(b)/(double)c;

Share on other sites
There is no difference. The following program gives identical output for both values of a:
#include <iostream>using namespace std;int main(){  double a = double(500000000)/9;  cout << fixed << a << endl;  int b = 500000000;  int c = 9;  a = double(b)/c;   cout << fixed << a << endl;}

Output is:
55555555.55555655555555.555556

Post your exact code and we might be able to see what's wrong.

Share on other sites
it is still truncating you to an int. You could actually need to have

double a = double(b) / double(c);

Share on other sites
If I'm not mistaken, only one side of / is required to be a double to make the result double (the compiler shouldn't be free to choose the result type).

From the standard:
Quote:
 --Otherwise, if either operand is double, the other shall be converted to double.

It is surprising to see that the OP's "wrong" result has been truncated "upwards".

Share on other sites
Yes it's true I've already tried converting both operands but I'm still getting the same result.

the exact code of my program is:

int b = 500000000;
int c = 9;
double a = double(b)/double(c);
a = double(500000000)/9;

And I check the value of variable 'a' in the debug mode of visual c++.

on line 3 , the value of a becomes 55555556.000000000
on line 4 , the value of a becomes 55555555.555555552

Does anyone have a clue?

even when I do:

double b = 500000000;
double c = 9;
double a = b/c;

the value of a is still 55555556.000000000

Share on other sites
That exact code gives equal output for both operations (55555555.555556) when compiled with g++. The discrepancy you see might be due to a debugger quirk. Try compiling and printing the result of the calculations through cout or printf.

By the way, what version of MSVC are you using? If it's a very old version (MSVC 6), there might be issues with nonstandard behaviour.

Share on other sites
Quote:
 Original post by TaxCollectorAnd I check the value of variable 'a' in the debug mode of visual c++.on line 3 , the value of a becomes 55555556.000000000on line 4 , the value of a becomes 55555555.555555552

Can't replicate with MVS2008. All debug outputs show correct value.

What does debug mode mean anyway, as tooltip, as watch, compiled as debug configuration?

Share on other sites
After experimenting with the code I've found that at the very start of the program the calculation is done as expected.

It is only after calling the function

D3DDEVTYPE_HAL,
hWnd,
D3DCREATE_SOFTWARE_VERTEXPROCESSING,
&d3dpp,
&d3ddev);

(where d3ddev is an LPDIRECT3DDEVICE9;
and d3dpp is a D3DPRESENT_PARAMETERS;)

that the problem arises. So maybe this problem should've been posted in the DirectX part of the forum but I did not know the problem was caused by directX when I first posted the question. Because I'm a novice in DirectX, I do not know how to solve the issue.

Share on other sites
I believe DirectX modifies the way the floating point processor works. There is a flag "D3DCREATE_FPU_PRESERVE" that can be used (somewhere; I don't use DirectX [smile]) to keep the original behaviour. See if this is the source of your problem.

1. 1
2. 2
3. 3
4. 4
frob
15
5. 5

• 16
• 12
• 20
• 12
• 19
• Forum Statistics

• Total Topics
632163
• Total Posts
3004514

×