formatFloat() fails for documented DBL_MAX

Started by
3 comments, last by iraxef 9 years, 4 months ago

According to the documentation on primitives, 1.7976931348623158e+308 is the maximum value of a double. However, formatInt(1.7976931348623158e+308, "e") will print "inf". (Doing some brute-force testing, if the latest digit there is anything higher than 5, "inf" will be the result.)

According to this C++ reference, the maximum double value is 1.79769e+308. Where did you get the value as documented? Could it be non-portable (e.g. works on Windows, but is technically higher than the max)?

Thank you.

Advertisement

The C++ reference you're looking at is only showing you what cout prints, but cout by default limits the number of decimal digits printed so it doesn't show all of them even though they are available.

I got the constants from here:

http://msdn.microsoft.com/en-us/library/6bs3y5ya.aspx

You can also see it another reference here:

http://en.wikipedia.org/wiki/Double-precision_floating-point_format

Most likely the problem is not really related to the constant itself, but rather due to numerical rounding errors during the conversion from text to binary done in asStringScanDouble in the compilation. I'll take a look at it.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

Confirmed that print(formatFloat(1.7976931348623158e+308, "e")); returns "2e+308" when locally modified asStringScanDouble to return strtod(string, NULL); instead of return value;

Would it make sense to use some of the platform ifdefs from as_config.h and use strtod on platforms where it's available?


double asStringScanDouble(const char *string, size_t *numScanned)
{
    char* endptr = NULL;
    double value = strtod(string, &endptr);

    if( numScanned )
        *numScanned = endptr - string;

    return value;
}

Or perhaps use the actual source (e.g. http://www.opensource.apple.com/source/tcl/tcl-10/tcl/compat/strtod.c)

I tried the source code from apple, but not even that is not capable of performing the conversion correctly.

I suspect actual implementation for strtod that MSVC uses takes advantage of the knowledge of the IEEE 754 standard and directly converts the string into the binary representation. That is the only way it could be done without getting rounding errors.

I'm not planning on going this far, so rather than changing the current implementation I just changed the manual to avoid giving that many digits.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

For what it's worth I tried strod() on linux and that worked fine (I imagine that may be a closer implementation to Apple's).

But I think your change is fine.

This topic is closed to new replies.

Advertisement