Jump to content

  • Log In with Google      Sign In   
  • Create Account

FREE SOFTWARE GIVEAWAY

We have 4 x Pro Licences (valued at $59 each) for 2d modular animation software Spriter to give away in this Thursday's GDNet Direct email newsletter.


Read more in this forum topic or make sure you're signed up (from the right-hand sidebar on the homepage) and read Thursday's newsletter to get in the running!


- - - - -

Linux x86-64 not loading or saving bytecode correctly.


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
20 replies to this topic

#1 urkle   Members   -  Reputation: 138

Like
0Likes
Like

Posted 23 July 2012 - 05:05 PM

I'm using Angelscript 2.22.1 in a game I'm porting to Linux.. The release game distributes JUST the compiled angelscript byte code, however this isn't working correctly on x86_64 linux. Here is the breakdown of what it's doing.

32bit linux loads the bytecode generated by the 32bit linux game binary,, does NOT load the bytecode generated by the 64bit binary
64bit linux does not load either bytecode correctly.

Does the bytecode depend on how the calls are registered? As some calls are wrapped (using the autowrapper) on x86_64 linux only. (they use Native calling on all other platforms). This is to work around the passing and returning of certain structs by value.

Sponsor:

#2 Andreas Jonsson   Moderators   -  Reputation: 3453

Like
0Likes
Like

Posted 23 July 2012 - 06:27 PM

Version 2.22.1 did not yet have a platform independent bytecode, this was only implemented in version 2.23.0.

In version 2.22.1 Linux 64bit should be able to load bytecode compiled on Windows 64bit and vice versa, as should Linux 32bit be able to load bytecode from Win 32bit and vice versa. Linux 64bit won't be able to load Linux 32bit bytecode, and vice versa.

The bytecode does not depend on the registered functions' calling convention, as long as they are registered with the same declaration, the bytecode loader should work correctly.

I'm not aware of any bugs in version 2.22.1 that causes bytecode not to load properly on Linux 64bit. The bug fixes that have been done in later releases are all related to the changes done in version 2.23.0 to make the code platform independent.

Would it be possible to narrow down the script that fails to load properly, so that I can try to determine the cause of the failure? Unfortunately I do not have access to a 64bit Linux environment to test the library on anymore, since Jeremy's not responding anymore, but depending on the situation I may be able to figure out the cause anyway.

May I ask what game it is?
AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

#3 urkle   Members   -  Reputation: 138

Like
0Likes
Like

Posted 23 July 2012 - 08:53 PM

I tried bumping to 2.23.1 or 2.24a and now I get new issues when saving.. I'm getting an offset != currOffset in asCWrite::AdjustGetOffset in 64bit!

Though running 32bit saves and loads the bytecode perfectly fine.. and the 64bit loads that bytecode fine..

Odd.

This is using the 2.24.0 tag from SVN.

Wait.. scratch that.. it's acting REAL funcy now in 64bit :(.. causing odd errors in game.. Ugh!!!!

it's not easy to see which exact source file is causing issues as the whole thing is compiled into one file via the main.cpp.

And I can send you a PM w/ the game name

#4 quarnster   Members   -  Reputation: 266

Like
0Likes
Like

Posted 23 July 2012 - 11:49 PM

Andreas, if its of any help debugging this issue, test_feature doesn't fail but reports the following on x86_64 OS X:

The saved byte code is not of the expected size. It is 1888 bytes
The saved byte code contains a different amount of zeroes than the expected. Counted 620
The saved byte code has different checksum than the expected. Got 0xCC85F81D
-- TestSaveLoad passed



#5 Andreas Jonsson   Moderators   -  Reputation: 3453

Like
0Likes
Like

Posted 24 July 2012 - 06:09 AM

Hi quarnster, this tells me that there is indeed something wrong in the 64bit save/load feature. I'll try to see if I can reproduce this on my Win64 machine. Hopefully it is the same problem that urkle is facing.
AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

#6 Andreas Jonsson   Moderators   -  Reputation: 3453

Like
0Likes
Like

Posted 24 July 2012 - 06:13 AM

This is using the 2.24.0 tag from SVN.


If you are going to use 2.24.0 then you should get release 2.240a. There was a rather serious bug in the 2.24.0 release causing memory invasions.
AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

#7 Andreas Jonsson   Moderators   -  Reputation: 3453

Like
0Likes
Like

Posted 24 July 2012 - 12:12 PM

I managed to get access to a 64bit Linux environment to try this on. Unfortunately I was not able to reproduce the problem, neither with version 2.22.1 nor with the latest revision from the SVN.

There are a few minor issues with the latest revision from the SVN that I'll check in tonight, but they are unrelated to bytecode saving/loading.



I will need more information to be able to fix this problem.

First, will you be working with the latest release or stick with version 2.22.1? For me the latest would be best, but I understand if you'd prefer to stick with the version already used by the game on other platforms.

Second, I really need for you to narrow down the script that is failing to load, so I can debug it in my own environment in order to figure out the problem. Try commenting out part of the script, then test the save/load part. If it still fails, comment out more of the script, it it doesn't fail, uncomment the script and comment the other part instead. With a few iterations like this you should be able to get a reasonably small script.

I understand the problem shows immediately as the script is loaded, i.e. the LoadByteCode() method returns an error, and not just when running the game, right? This ought to make it easier to pinpoint the problem.
AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

#8 Andreas Jonsson   Moderators   -  Reputation: 3453

Like
0Likes
Like

Posted 24 July 2012 - 04:13 PM

The minor issues that I found in the tests along with the patches provided by quarnster in the other thread have been checked in to revision 1370.
AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

#9 Andreas Jonsson   Moderators   -  Reputation: 3453

Like
0Likes
Like

Posted 25 July 2012 - 10:04 AM

I backported the tests that I have for the latest WIP to version 2.22.1 and some of the problems that have been fixed in the latest releases were already pre-existent in version 2.22.1 even before the major changes in 2.23.0.

The situations that I found out that fails on 2.22.1 are the following:

- Arrays initialized with function pointers (fails to properly store the function pointers that the array is initialized with)
- Non-shared classes that inherit from shared classes (fails when loading the second module with the same shared class)

However, both of these problems also occur on 32bit so I do not believe they are related to the problem you're facing with Linux 64bit. I won't work on fixes for these problems in version 2.22.1, unless it is determined that they really are what is causing your problem.
AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

#10 m4ttbush   Members   -  Reputation: 125

Like
0Likes
Like

Posted 26 July 2012 - 12:51 AM

Hey Andreas,

I've managed to isolate the problem!
Happens on both Windows and Linux 64 bit with 2.24.0a
Looks like it's caused by an enum followed by a non-primitive type in a script function...:
enum TestEnum
{
  TestEnum_A
}
class NonPrimitive
{
}
// Problem function
void Foo( TestEnum e, NonPrimitive o )
{
}
void main()
{
  NonPrimitive o;
  Foo( TestEnum_A, o ); // Crashes saving bytecode for where it's called
}
"NonPrimitive" can be value or reference, script or application class and it will crash, but primitive types work fine.

Looks like it breaks in as_restore.cpp around line 3369 when it gets to the enum parameter.
Maybe some kind of discrepancy in how enum parameters are treated?
for( asUINT p = 0; p < calledFunc->parameterTypes.GetLength(); p++ )
{
  if( offset <= currOffset ) break;
  if( calledFunc->parameterTypes[p].GetObjectType() ||
	calledFunc->parameterTypes[p].IsReference() )
  {
	numPtrs++;
	currOffset += AS_PTR_SIZE; // offset is 1 on 32 bit and 64 bit, but this causes currOffset to be 2 on 64 bit
  }
  else
  {
	asASSERT( calledFunc->parameterTypes[p].IsPrimitive() );
	currOffset += calledFunc->parameterTypes[p].GetSizeOnStackDWords();
  }
}

Edited by m4ttbush, 26 July 2012 - 06:45 AM.


#11 Andreas Jonsson   Moderators   -  Reputation: 3453

Like
0Likes
Like

Posted 26 July 2012 - 09:38 AM

Great!

I'll give this a try, and provide a fix as soon as possible.
AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

#12 Andreas Jonsson   Moderators   -  Reputation: 3453

Like
0Likes
Like

Posted 26 July 2012 - 11:35 AM

I've reproduced the problem on the latest WIP and will be working on the fix. However, the same scenario is working OK in version 2.22.1, so it doesn't seem to be the cause of the problem that urkle reported.
AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

#13 Andreas Jonsson   Moderators   -  Reputation: 3453

Like
0Likes
Like

Posted 26 July 2012 - 11:53 AM

The fix in 2.24 is easy.

for( asUINT p = 0; p < calledFunc->parameterTypes.GetLength(); p++ )
{
  if( offset <= currOffset ) break;
//  if( calledFunc->parameterTypes[p].GetObjectType() ||      <-- exchange this line for the following
   if( !calledFunc->parameterTypes[p].IsPrimitive() ||
	    calledFunc->parameterTypes[p].IsReference() )
  {
	    numPtrs++;
	    currOffset += AS_PTR_SIZE;
  }
  else
  {
	    asASSERT( calledFunc->parameterTypes[p].IsPrimitive() );
	    currOffset += calledFunc->parameterTypes[p].GetSizeOnStackDWords();
  }
}

The function AdjustGetOffset() doesn't exist in version 2.22.1. It was introduced in version 2.23.0 so the problem in 2.22.1 is definitely a different one.

Regards,
Andreas
AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

#14 Andreas Jonsson   Moderators   -  Reputation: 3453

Like
0Likes
Like

Posted 26 July 2012 - 03:55 PM

I've checked in this fix in revision 1372.
AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

#15 m4ttbush   Members   -  Reputation: 125

Like
0Likes
Like

Posted 26 July 2012 - 07:27 PM

Thanks for the prompt fix Andreas!

The bytecode saves out ok from 64 bit now.
Unfortunately I'm still having issues with loading bytecode on 64 bit.

The bytecode seems to load ok and it runs, but when it executes a function with an enum parameter, the other parameters get mangled.
For example, I have a script function:
void change_state(state_types new_state, int new_face)
When the scripts are compiled on x64 it works fine, but when the bytecode is loaded again, the second parameter becomes the same value as the enum.
Replacing state_types with int then casting it back to an enum to assign it fixes the problem.
There was another case with a few enums and a string that caused a crash in the string assignment operator when the function was called. This was also fixed by replacing the enums with ints.
The function I was originally having problems with also causes problems, so I'm assuming this is the same problem but manifesting itself in a different way.

I'll see if I can isolate the problem again, I suspect it will just be a matter of reloading the bytecode in the test I posted earlier.
Edit: Yep, using the same script as before but saving then loading the bytecode before executing it causes a crash, but it works if the enum is swapped for an int.

Thanks for your help!

Edited by m4ttbush, 26 July 2012 - 07:51 PM.


#16 Andreas Jonsson   Moderators   -  Reputation: 3453

Like
0Likes
Like

Posted 26 July 2012 - 07:53 PM

The same fix needed to be applied in the asCReader::AdjustGetOffset() method too. I checked in that fix in revision 1373 now.

Regards,
Andreas
AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

#17 m4ttbush   Members   -  Reputation: 125

Like
0Likes
Like

Posted 26 July 2012 - 08:26 PM

Thanks Andreas,
I was having the same issue with revision 1373, but I made the same changes in asCWriter::CalculateAdjustmentByPos() and asCReader::CalculateAdjustmentByPos()
And everything seems to be working now!
Does that sound right?

Thanks!

#18 Andreas Jonsson   Moderators   -  Reputation: 3453

Like
0Likes
Like

Posted 27 July 2012 - 07:57 AM

Yes, it sounds correct. Hopefully that was the last of them.

I'll add some further test cases to try to catch these problems and avoid that they re-occur with future changes.
AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

#19 Andreas Jonsson   Moderators   -  Reputation: 3453

Like
0Likes
Like

Posted 27 July 2012 - 07:03 PM

I checked in these last changes in revision 1374. Let me know if you find any further problems with the latest release.
AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

#20 urkle   Members   -  Reputation: 138

Like
0Likes
Like

Posted 28 July 2012 - 07:37 AM

Andreas you are AWESOME! Thanks so much for turning around this bug fix (and every other bug fix you've ever turned around for me in the past ! )




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS