Archived

This topic is now archived and is closed to further replies.

Ishan

possible bug in ms visual studio?

Recommended Posts

I'm using microsoft visual studio to do some 3D math libraries, and I spent the better part of the night tracking down a wierd bug. Basically, my compiler maintains that casting the result of a complex sqrt() to float makes it != to itself. In other words, if a function to take the magnitude of a vector returns a float, wierd shit happens. Check out this example code:
  
#include <iostream>
#include <cmath>
using namespace std;

float a[3] = {1,2,3};

float mag(float * ptr){
	return sqrt(ptr[0] * ptr[0] + ptr[1] * ptr[1] + ptr[2] * ptr[2]);
}

double mag2(float * ptr){
	return sqrt(ptr[0] * ptr[0] + ptr[1] * ptr[1] + ptr[2] * ptr[2]);
}

void main(){
	float c = mag(a);
	float d = mag(a);
	cout << "assigned mag(a) to float c and d" << endl;
	if(c == d){
		cout << "c and d are equal" << endl;
	} else {
		cout << "c and d are not equal" << endl;
	}
	if(mag(a) == mag(a)){
		cout << "mag(a) equals mag(a)" << endl;
	} else {
		cout << "mag(a) does not equal mag(a)" << endl;
	}
	if(mag2(a) == mag2(a)){
		cout << "mag2(a) equals mag2(a)" << endl;
	} else {
		cout << "mag2(a) does not equal mag2(a)" << endl;
	}	
}
  
Under microsoft visual studio .net, it prints: assigned mag(a) to float c and d c and d are equal mag(a) does not equal mag(a) mag2(a) equals mag2(a) In other words, calling the same function on the same data twice does not yield equal results, if the function returns a float. An identical function which returns a double does not experience this problem. Is this a bug in the compiler or library? Am I just doing something stupid? The only possibility that I could see is that there is some kind of safety mechanism which notices when data is lost (e.g. a double is converted to a float) and sets the two pieces of data to be not equal, in case we had a situation like 1.00000...1 and 1.000000....2, where .... is more precision than a float can handle, and less than a double. [edited by - Ishan on April 28, 2003 3:44:22 AM]

Share this post


Link to post
Share on other sites
Yes, you shouldn't ever compare floats or doubles with == for equality.

Instead, do:

bool IsEqual(float lhs,float rhs,float tolerance=1e-4)
{
return fabs(lhs-rhs) < tolerance;
}

Note: VS 2003 finally has non-retarded abs,sin,cos,sqrt routines!

[edited by - sjelkjd on April 28, 2003 4:03:09 AM]

Share this post


Link to post
Share on other sites
quote:
Original post by sjelkjd
Yes, you shouldn''t ever compare floats or doubles with == for equality.

Instead, do:

bool IsEqual(float lhs,float rhs,float tolerance=1e-4)
{
return fabs(lhs-rhs) < tolerance;
}

Note: VS 2003 finally has non-retarded abs,sin,cos,sqrt routines!

[edited by - sjelkjd on April 28, 2003 4:03:09 AM]


Very interesting, thanks.

Share this post


Link to post
Share on other sites
quote:
Original post by petewood
does your compiler give any warnings?


Yeah, it says "conversion from double to float, possible loss of data"...but shouldn''t it be the same data lost every time?

Share this post


Link to post
Share on other sites
This isn''t a bug as such. This is more to do with the way floating point values are handled by the CPU. On the Intel chips, the values internally are stored in double format by default.
In this program, the compiler needs to create a temporary variable to store the result of the first "mag (a)" in the line "if(mag(a) == mag(a))", this temporary is of type float. The compiler generates this code sequence:

push offset a
call mag
store floating point value at top of fp stack to temporary variable

The result of the function call is returned on the top of the floating point stack - so it''s *always* a double, even if you declare the function a float. The conversion to float occurs when the returned value is saved to a variable.
The next call to "mag (a)" is the same as the first but the result is never saved to memory - it doesn''t need to be.
The comparison then becomes a compare between the temporary variable and the value on the top of the floating point stack - effectively a comparison between a float (the temporary) and a double (the fp stack) which will yield a non-equal result in this instance as the float version has lost some precision.

The first comparison works because both values have been truncated when saved to the variables, the third works because the temporary value is a double and so no precision is lost.

To solve this, either save the returned values to variables before the comparison (slower - two dwords are written) or cast the first call to a double (not quite as slow - a qword is written instead of a dword):

if((double)mag(a) == mag(a))

This forces the temporary value to be a double and not a float. Casting to float doesn''t work (it''s ignored).

Saving the result of the first call on the FP stack is not an option because the compiler can''t garantee that the function ''mag'' doesn''t use all the FP stack and hence lose the result of the first call (in this case it probably doesn''t though).

Skizz

Share this post


Link to post
Share on other sites
This explains why I couldn''t find my last big "bug". :D
Interesting post indeed...

"He who hasn''t hacked assembly language as a youth has no heart. He who does as an adult has no brain."

Share this post


Link to post
Share on other sites
quote:
Original post by Skizz
This isn''t a bug as such. This is more to do with the way floating point values are handled by the CPU. On the Intel chips, the values internally are stored in double format by default.


Actually, it''s a little more complicated than that. x87 uses 80 bits of internal precision. If you use SSE, you only get 32 bits though(or 64 if you use double precision). That might sound weird, but remember that intel is trying to get everyone to use SSE2 instead of x87(even for scalar ops), since the p4 x87 is so weak.

Share this post


Link to post
Share on other sites
quote:

Actually, it''s a little more complicated than that. x87 uses 80 bits of internal precision.


This is wrong - sort of - the FPU is running 64 bits precision in this application since the call to mag2 works (unless the value returned is exactly representable in 64 bits - unlikely). By your argument, it would be comparing a 64 bit float with an 80 bit float which would almost certainly be unequal (the reason the call to mag fails).
The FPU can do 80 bit calculations if you tell it to, but does 64 by default. At least the MS CRT libraries set it to 64 bit precision by default. To get 80 bit precision using VC you need to link to LIB/FP10.OBJ (which must appear before other libraries on the linker command line).

Skizz

Share this post


Link to post
Share on other sites