# Constant integer corruption with VS2010

## Recommended Posts

Hello!

After upgrading to r2418, we again found a bug related to constant handling, it seems.

Test case:

const int64 a = 4287660160; print(formatInt(a, "0h", 8)); // ffd08080 (WRONG)
int64 a = 4287660160; print(formatInt(a, "0h", 8)); // ffd08080 (WRONG)
const int64 a = 4287660160.0f; print(formatInt(a, "0h", 8)); // ffd08000 (WRONG)
int64 a = 4287660160.0f; print(formatInt(a, "0h", 8)); // ff908000 (CORRECT)

I cannot reproduce the issue with a linux build with gcc 4.2 nor gcc 7. It seems to happen exclusively on VS2010.
This broke our PNG map loading scripts because these compare hex literals against colors.

I think we didn't do anything wrong on our side? I hope it's not too hard to fix.

Edited by Asu

##### Share on other sites

That is very odd, more so that it only happens on MSVC, as it doesn't look like the type of problem that would only happen on one compiler/platform.

I'll investigate it.

##### Share on other sites

I haven't been able to reproduce the problem you reported. I use MSVC2015, though I sincerely doubt it would be different with MSVC2010.

I did the test like this:

r = ExecuteString(engine,
"const int64 a1 = 4287660160; assert(formatInt(a1, '0h', 8) == 'ff908080'); \n"
"int64 a2 = 4287660160; assert(formatInt(a2, '0h', 8) == 'ff908080'); \n"
"const int64 a3 = 4287660160.0f; assert(formatInt(a3, '0h', 8) == 'ff908000'); \n" // due to float precision the result is different, but correct
"int64 a4 = 4287660160.0f; assert(formatInt(a4, '0h', 8) == 'ff908000'); \n" // due to float precision the result is different, but correct
"const int64 a5 = 4287660160.0; assert(formatInt(a5, '0h', 8) == 'ff908080'); \n"
"int64 a6 = 4287660160.0; assert(formatInt(a6, '0h', 8) == 'ff908080'); \n"
);
if (r != asEXECUTION_FINISHED)
TEST_FAILED;	

Note, 427660160.0f is not equal to 4287660160. The float type has only precision enough for 6 digits, so the 427660160.0f actually becomes 4287660032. (This of course depends on the way the float is represented in memory, but IEEE 754 standard is the most common on the CPUs).

##### Share on other sites

Weird... Our map loading is broken only on our Windows builds, but not on the Linux builds. I guess we will investigate on our side.

## Create an account

Register a new account

• 15
• 9
• 13
• 41
• 15