Jump to content
  • Advertisement
Asu

Constant integer corruption with VS2010

This topic is 388 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello!

After upgrading to r2418, we again found a bug related to constant handling, it seems.

Test case:

const int64 a = 4287660160; print(formatInt(a, "0h", 8)); // ffd08080 (WRONG)
int64 a = 4287660160; print(formatInt(a, "0h", 8)); // ffd08080 (WRONG)
const int64 a = 4287660160.0f; print(formatInt(a, "0h", 8)); // ffd08000 (WRONG)
int64 a = 4287660160.0f; print(formatInt(a, "0h", 8)); // ff908000 (CORRECT)

I cannot reproduce the issue with a linux build with gcc 4.2 nor gcc 7. It seems to happen exclusively on VS2010.
This broke our PNG map loading scripts because these compare hex literals against colors.

I think we didn't do anything wrong on our side? I hope it's not too hard to fix.

Edited by Asu
added revsion number

Share this post


Link to post
Share on other sites
Advertisement

That is very odd, more so that it only happens on MSVC, as it doesn't look like the type of problem that would only happen on one compiler/platform.

I'll investigate it.

Share this post


Link to post
Share on other sites

I haven't been able to reproduce the problem you reported. I use MSVC2015, though I sincerely doubt it would be different with MSVC2010.

I did the test like this:

r = ExecuteString(engine,
     "const int64 a1 = 4287660160; assert(formatInt(a1, '0h', 8) == 'ff908080'); \n"
     "int64 a2 = 4287660160; assert(formatInt(a2, '0h', 8) == 'ff908080'); \n"
     "const int64 a3 = 4287660160.0f; assert(formatInt(a3, '0h', 8) == 'ff908000'); \n" // due to float precision the result is different, but correct
     "int64 a4 = 4287660160.0f; assert(formatInt(a4, '0h', 8) == 'ff908000'); \n" // due to float precision the result is different, but correct
     "const int64 a5 = 4287660160.0; assert(formatInt(a5, '0h', 8) == 'ff908080'); \n"
     "int64 a6 = 4287660160.0; assert(formatInt(a6, '0h', 8) == 'ff908080'); \n"
    );
if (r != asEXECUTION_FINISHED) 
   TEST_FAILED;	

Note, 427660160.0f is not equal to 4287660160. The float type has only precision enough for 6 digits, so the 427660160.0f actually becomes 4287660032. (This of course depends on the way the float is represented in memory, but IEEE 754 standard is the most common on the CPUs).

 

 

 

Share this post


Link to post
Share on other sites

Weird... Our map loading is broken only on our Windows builds, but not on the Linux builds. I guess we will investigate on our side.

Share this post


Link to post
Share on other sites

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!