# AngelScript 1.10.0 WIP 6 (2004/10/31)

This topic is 4831 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

Dear subscribers, another update has been made. There's been a few changes to the interface to make it more consistent and easier to use. However, after receiving user feedback, I have kept backwards compatibility with 1.9.x. Though this backwards compatibility is only available if the library and host application is compiled with the preprocessor define AS_DEPRECATED. The methods that have changed are:
• RegisterObjectBehaviour(), it only accepts true object behaviours now, i.e. the ones that you register with the first parameter naming the object. Another method, RegisterGlobalBehaviour(), has been created for the other behaviour functions.
• GetFunctionName(), GetFunctionDeclaration(), GetGlobalVarName(), GetGlobalVarDeclaration(), GetImportedFunctionDeclaration(), and GetExceptionString() has changed to return const char * instead so that you no longer have to allocate the buffer before. The pointer that is returned is shared between these functions so you shouldn't store it for later use.
• All methods that return a string, also take an optional parameter that allow the host to get the length of the returned string directly instead of calling strlen().
I've also started to add support for multithreading. Currently this only works on Win32 and with MSVC++, but I plan on expanding this to Linux (I'll need help on that one) and MinGW. I got some strange problem when trying to support multithreading with MinGW, the code compiled ok, but when testing it would crash at inlogical places, places that haven't been changed. I think it is somehow connected to including windows.h or STL map in the code. Anyway, the support that was added for multithreading was:
• All reference counting is now threadsafe
• asGetActiveContext() is working correctly
• The shared string (mentioned above) is working by keeping a string buffer for each thread
• Created a new global function asThreadCleanup() that should be called before terminating any secondary threads that have accessed the library. If it is not called the local data allocated for the thread will be cleaned up at application exit. Note, it is not necessary to call this function in a single threaded environment.
It should now be possible to use AngelScript and access the engine and modules from multiple threads at the same time. It still very experimental, but at least rudimentary support for multithreading is now available. Please let me know if you find some trouble or wish to help me provide support for Linux and MinGW as well. I've also made a bug fix in the optimization. I had mistakedly optimized subtractions as if they were associative, this could make calculations with subtractions give the wrong result. Many thanks to Lennart Denninger for helping me find that bug. Regards, Andreas Jönsson Author of AngelScript [Edited by - WitchLord on October 31, 2004 11:05:15 AM]

##### Share on other sites
One more update!!!! Way to go......glad I still have'nt updated from 1.7.x though. will take the giant step when 1.10.0 comes out

##### Share on other sites
EddHead: The more you wait, more work will be necessary to do the upgrade. Although for this latest version I'm trying to keep backwards compatibility.

##### Share on other sites
I didn't compile the library in release mode before uploading the latest WIP. Lennart let me know that there is some problem when doing so, but fortunately the fix is easy: Just turn on language extensions in Visual C++.

I will try to make the library fully working without Microsoft's language extensions.

##### Share on other sites
I've uploaded the second WIP of version 1.10.0.

This version brings a much improved ExecuteString() method. The improvements made to the function is:
- Calls to ExecuteString() can be recursive, i.e. ExecuteString() can call a function that in turn calls ExecuteString()
- The application can pass its own context to ExecuteString() for preparation and execution (or just preparation)
- ExecuteString() should work with multiple threads (I haven't tested this yet, but I don't see why not)
- Only one call to ExecuteString() and retrieve the asIContext for exception handling (pass a pointer to a variable that will receieve the context)
- GetContextForExecuteString() is deprecated.

The old functionality is still available by defining AS_DEPRECATED when compiling the library

I've improved the multithread code so that it now works with MinGW as well. I removed the include of windows.h by locally declaring the few functions needed for thread handling. std::map had to be replaced with my own code as I found that the STL library isn't all that well-working as it ought to be. Microsoft's implementation of the STL library doesn't compile when you turn off the language extensions, and MinGW's implementation causes run-time problems with other template codes. My own map class (found in as_map.cpp) isn't quite as well structured as STL's map, but it works. However, I will improve upon it and start using it for other purposes as well, for example for mapping strings to classes, which should improve script compile times as well.

Yes, I know of STLPort which is supposedly much better than any STL library that comes with the compilers, but I don't want to force people to download another library just to compile mine. I have been thinking about rewriting the interface of my classes to match those of STL though. That would make it possible to replace my implementation and save some bytes in the final application.

I also fixed a bug with SetArguments(). It didn't allow the application to pass a pointer to the memory where the script function should return application registered classes. Thanks goes to Thomas Suter for helping me discover that bug.

Regards,
Andreas Jönsson
Author of AngelScript

##### Share on other sites
I'm actually using 1.9.2 at the moment, but didn't AS used to protect scripts from dereferencing null pointers? In the past I thought that AS just quietly aborted the script call if the script tried to dereference null, but now it goes ahead and crashes. Or am I wrong?

##### Share on other sites
It could be a bug. Can you show me an example script that crashes?

##### Share on other sites
Apparently it is only a problem with ExecuteString:

Data* p = 0;
p->GetId();

Tested it with normal scripts and they aborted fine...

##### Share on other sites
It is a bug.

What is happening is that an exception is raised because of the null-pointer access. The crash happens when SetInternalException() is called. Because the functionID for ExecuteString is 65533 there is an illegal memory access with when calling GetScriptFunction(functionID).

If you're pressed for a bug fix you could simply check for this case and skip that call.

The bug was introduced in version 1.9.1 when I took out the line numbers from the bytecode and stored them in a separate array.

##### Share on other sites
Here's the complete list of bug fixes available in version 1.9.2a and 1.10.0 WIP 3:

- bug fix: Subtraction was optimized as if it was associative (thanks Lennart Denninger)
- bug fix: SetArguments() failed for script functions that returns registered object types by value. (thanks Thomas Suter)
- bug fix: The application would crash if an exception was raised in ExecuteString() (thanks Joe "Desdemona" Wright)
- bug fix: GetFunctionName() and GetFunctionDeclaration() now works for the ExecuteString() function id as well.
- bug fix: The compiler didn't warn when passing uninitialized variables to function parameters (thanks Alain "abrken" Bridel)
- bug fix: Constructors with parameters could cause assertion failure in GenerateExceptionHandler(). (thanks Alain "abrken" Bridel)

I've started work on incorporating the x86 assembler VM that Andres Carrera (Lioric) sent me. I hope to have this done by this weekend. Then we will see what kind of performance improvement we will really get.

##### Share on other sites
The VM would be a great AddOn, is there a possibility we could get a compiler like program which could be installed in VC++ (or any other IDE) to point out errors in the script during a pre-build process?

This way we make sure the code starts with a clean script.

##### Share on other sites
If I had the time I would write one, but I don't have that time.

It shouldn't be that difficult though. Write a small console program that registers all the same functions and prototypes as your game, but use null pointers instead of the real function pointers. Then compile the script and make the necessary verifications of the scripts. If the script doesn't compile correctly or is missing a necessary function you should end the application with returning non-zero from main(). That way nmake can recognize the compiler error and stop the build.

I should think that this would be the work of at most a couple of hours for a specific case like this.

To make it generic, using configuration files, etc, it would require a lot more time.

##### Share on other sites
Hi

What is wrong with the actual built-in compiler? it has all the messages you will need in the build process and i dont see it why doing an external compiler will be better that using a code syntax library inside your editor

It will be better doing an stand alone compiler if you want to create a stand alone application with AS, extending AS from an embedable lib to a programing language tool

I think we better should focus on the debugger features of the lib, improving them and adding them

Take maxscript as an example, its a full featured script language, you can control over 99% of 3dsMax, but its a mess to debug it

I will work on adding debug hooks to AS very soon

Lioric

##### Share on other sites
I've finally been able to get Lioric's assembler VM working with the latest WIP of 1.10.0. So far the increase in performance is only about 25%. This is because the new bytecodes introduced in version 1.9.x still haven't been converted to assembler, these bytecodes are the ones that are used the most. Once they are converted to assembler I expect the performance to be at least doubled.

I will hopefully be able to release another WIP this weekend, with the assembler VM working (although not fully optimized).

Those who are using MinGW, Dev/C++, DJGPP, GNUC, etc still won't be able to use the new VM though, as these compilers don't understand the intel syntax for inline assembly. If anyone feel up to converting the intel syntax to the AT&T syntax used by GNUC I would appreciate it very much.

Just thought I'd let you know how the work is going.

##### Share on other sites
Hi

you know im here if you need help converting the new bcodes to asm

why so low improve in perfomance, keep in mind that if you benchmark all the script lib, from compile to execute, you are including the symbol relocation in the benchmark, you should time only the abstract machine execution, and because the vm's differences, the performance increase should be in the order of x not in %, like 1.5 x or 2x

Best Regards,

Lioric

##### Share on other sites
Yes, I know you have offered your help. And you'll get your chance when I release the next WIP.

I only time the actual execution, so I'm sure the numbers are right. But don't worry, the improvement will go up as the rest of the bytecode instructions are converted to asm. If you're interested, you can download the performance test that I make in the latest WIP from the site.

The low performance improvement can probably be explained with that most of the bytecodes that are currently run are actually written in C++. Thus they only benefit from the relocation table that you created. Some bytecodes had changed so much with 1.9.x that I simply put in the C++ code instead of your assembler code, others are completely new and substitute the bytecodes that you had written in assembler.

I have concentrated on getting things to work and cleaning up the code. I haven't actually done any conversion to assembly myself. Before I release the next WIP I will make the saving/loading of bytecode work again with the new assembly VM.

Regards,
Andreas

PS. I'd like to thank you once more for this great contribution.

##### Share on other sites

I've uploaded the latest WIP of 1.10.0. This version includes Andres Carrera's assembler optimized virtual machine. It's not fully optimized yet though, as he based the VM on AngelScript version 1.8.2, and there's been a lot of changes to the VM since then. The assembler VM is currently slightly less than 1.5x faster than the C++ version, I suspect that this will go up as the VM is optimized more over time.

It's not an easy task to optimize assembler though. Sometimes, when I converted a bytecode implementation from C++ to assembler the performance decreased as well, showing that at times it is simply easier to let the C++ compiler do the optimization for you. And sometimes when substituting two assembler instructions for one that did the same thing the performance also decreased, even though logically it should have improved. This is probably due to the processor's ability to pipeline instructions, and even parallel execution in some cases.

To use the assembler VM you need to compile the library with the flag USE_ASM_VM defined. Note that the assembler VM is only available for compilers that understand the intel assembler syntax. Once the assembler VM is more stabilized I will work on converting the code to AT&T syntax for the GNUC based compilers. Also note that the assembler VM hasn't been fully tested yet, as I've found that many of the instructions are never executed by my test cases.

I will now concentrate on implementing support for native arrays, and also writing testcases to cover the rest of the bytecode instructions.

Regards,
Andreas Jönsson
Author of AngelScript

##### Share on other sites
Hm, not sure if this can be called a bug, but the following just happened to me. Something seemed to go wrong, but I couldn't find it as no error messages were given or anything.

This is some code I wrote to test this (Class 1 doesn't do anything but contain the static Method, DebugStream just does a printf):
	DebugStream ds;	asIScriptEngine* scripts=asCreateScriptEngine(ANGELSCRIPT_VERSION);	//Error here, of course, class not defined	scripts->RegisterGlobalFunction("Class1* GetClass1()",asFUNCTION(Class1::Get),asCALL_CDECL);	scripts->RegisterObjectType("Class1",0,0);	char* testcode="gjeskjhgskjegh";	scripts->AddScriptSection("test","test",testcode,strlen(testcode));	scripts->Build("test",&ds);	scripts->ExecuteString("test","gjngksejng",&ds);	scripting->Release();

Now, I know I should have checked the return value of the Register-stuff for errors, I just didn't because I was only going to test some stuff, but the problem is that after the RegisterGlobalFunction fails, all the rest just seems to fail silently.
If I switch around the declarations into correct oder, it works just as it should and complains that it can't compile nor execute that gibberish.

I admit that code like this shouldn't make it into any real programs, but some kind of error message would be nice, at least. Or is there something completely different that I am doing wrong?

##### Share on other sites
AHHHHHHHHHHHHHHH!

I just spent the last hour hour trying to figure out why I couldnt get some simple scripts to work.

I had forgotten to remove the "Data* p = 0; p->GetId();" (previous post) from one of the related scripts and it was causing AS to silently abort....

twitch...

I think I shall take this moment to properly display notifications when AS experiences an exception... Something that should have been done long ago... hehe

##### Share on other sites
Quote:
 Original post by WuntvorI admit that code like this shouldn't make it into any real programs, but some kind of error message would be nice, at least. Or is there something completely different that I am doing wrong?

You should AT LEAST test the return value from ExecuteString, I'm sure it gives asINVALID_CONFIGURATION witch means : "Hey man, I cannot compile your code since you have made something wrong with the library BEFORE compiling".

AbrKen

##### Share on other sites
Quote:
 Original post by abrkenYou should AT LEAST test the return value from ExecuteString, I'm sure it gives asINVALID_CONFIGURATION witch means : "Hey man, I cannot compile your code since you have made something wrong with the library BEFORE compiling".

Hm, okay, guess that makes sense.. I was only checking the outstream as I thought all errors would be printed there, guess I'll be a lot more rigorous with return values in the future. Still, would it be possible to keep the engine in a usable state even after an error like that or at least also print a visible message when it happens? It did confuse me a lot until I finally figured out what was going wrong.

##### Share on other sites
Both Build() and ExecuteString() will return asINVALID_CONFIGURATION, in this case. No message is output to the output stream as the compilation isn't even begun.

I could possibly allow AngelScript continue to work even when the configuration is invalid, but it might produce more difficult to find errors further on. This is why I decided to simply disable everything if the application configures the engine incorrectly. By disabling the engine you can be sure that the configuration is completely correct when it works.

I will add the "Invalid configuration" message to the outstream again.

##### Share on other sites
It's a great day for AngelScript. WIP 5 has been released with first ever support for native arrays, which I'll describe in detail further down. First the list of changes in this WIP:

* The C++ VM was optimized using the ideas introduced with the ASM VM. Because of the optimizing features of the MSVC compiler the C++ VM is now even faster than the ASM VM (about 10%). Later on the ASM VM will be optimized further as it is possible to remove quite a few instructions from each bytecode implementation even after the C++ compilers optimization. The ASM VM has potential to become twice as fast as the C++ VM, but 25% faster ought to be a realistic number.
* The message "Invalid configuration" is now written to the output stream again if some of the registration functions failed. People kept forgetting to check the return codes, and got confused as to why the scripts didn't work. This might help pointing them in the right direction.
* The operator 'xor' has been added for boolean types. An alias '^^' is also available. The operator does the exact same thing as '!=' does for boolean types, but it might be easier to understand in complex boolean expressions.
* Native support for single dimensional arrays of any type has been added. They work perfectly for scripts, but cannot be passed to or from the application yet. The next WIP will complete this support.

That's what has changed. Now here is the detailed explanation for how the arrays work.

An array variable is declared as followed:

int[] a; // empty array of integers
object[] b(5); // 5 objects initialized with the default constructor

The array's elements must be initialized manually at the moment:

b[0] = GetObject("mace");
b[1] = GetObject("hammer");
...

An array can be passed by value or by reference to functions, but the internal array buffer is shared so it doesn't really make a difference except for a slight overhead of initializing the other array object when passed by value.

void TestArray()
{
int[] a(1);
a[0] = 1;
ModifyArray(a);
// a[0] will now be 0, even though the array object was passed by value
}

void ModifyArray(int[] a)
{
a[0] = 0;
}

The arrays have a default method length() that returns the number of elements in the array.

int[] array;
uint a = array.length();

The element access is out-of-bounds checked, so an exception is thrown if the script tries to access elements outside the allocated array.

int[] array(10);
array[10] = 3; // Throws "out of bounds" exception

Let's see what more is needed to explain. The array indices are 0 based. You can declare an array of pointers, but you can't declare a pointer to an array.

The object's registered destructor and assignment operator can't be virtual methods on GNUC based compilers, yet. MSVC should be able to handle virtual methods.

The object's constructor can not be a virtual method on any compiler as one of it's task is to initialize the virtual function table for the object.

What is still missing in the support for arrays:

* Multi dimensional arrays. The script compiler may crash if you try to declare these at the moment. If the compiler doesn't crash the VM will. It hasn't been tested. I will probably not include support for multi dimensional arrays in version 1.10.0, maybe in 1.10.1. If I don't I will at least make sure the compiler gives the proper error messages, instead of crashing without warning.
* Automatic initialization of arrays in declarations: int[] a = {23, 23, 4, 12, 5};
* Temporary array objects in expressions: Function(int[](10));
* Cleaning up the code
* A C++ interface that allow the application to interact with script arrays
* I will try to allow the application to extend the native array object with more methods and operators, e.g. resize(), copy(), push_back(), +, etc.
* Support virtual destructor and assignment operators for the contained objects on all compilers

In the future I will also allow the application to register specialized array objects that will override the built-in array. The benefit of this is that the application can have a much cleaner interface with the script when working with a specialized array object, it would for example be possible to register std::vector. The script writer wouldn't know the difference.

I think that is all for now. Please let me know if you find any problems, or have any suggestions on how to improve the array support.

Regards,
Andreas Jönsson
Author of AngelScript
www.AngelCode.com

##### Share on other sites
The sixth WIP has been released. The changes for this version include:

• Minor changes to make compilation on Linux smoother (thanks Johannes Plass)
• Separate output files for release and debug mode (thanks Dan Royer for that suggestion)
• Inclusion of std::vector add-on that allows for simple registration of any std::vector object (implementation by Anthony "Deyja" Casteel). Note: MSVC6 seems to have trouble with vectors of vectors, e.g: vector< vector<int> > doesn't work. On GNUC it works though.
• It is now possible to register array objects, overriding AngelScript's native array for that type. This is the only way the application can interact with arrays in scripts. The array objects are registered as normal objects, except their data type should include the [] type modifier, e.g: int[].
• Multidimensional arrays are now working as well
• Array elements are copied on assignment with the native array.
• bug fix: Constants weren't correctly converted to references when passed as function arguments (thanks Anthony "Deyja" Casteel for detecting that one)
• bug fix: Switch case didn't treat 8 and 16 bit integer types correctly (thanks Alain "abrken" Bridel for spotting that)
• bug fix: Exception handler wasn't working correctly

There is still a little work left before I can release the final version of 1.10.0.

• Native arrays currently don't work on GNUC. It is possible to override the native array though.
• Registration of arrays of pointers probably don't work at the moment.
• The engine should report configuration error if an application function is registered that tries to interact with the native array.
• I need to verify that native arrays work with saved/loaded bytecode.

Once that has been done I will take care of the compiler performance problems that have been reported lately. I also want to make a much improved interface to script functions. I will also improve the rules implicit conversion.

In parallel with that I plan to start working on AngelScript 2. AngelScript 2 will change the way objects are handled by the script engine, which will be more like that of Java, with that pointers will no longer be supported. These changes will improve the library in several ways, where the two most important ones are a much cleaner syntax for script writers, and a library that is much easier to maintain and improve.

However AngelScript 1.x.x will continue to exist and I will continue supporting it with bug fixes and minor enhancements.

Regards,
Andreas Jönsson
Author of AngelScript

##### Share on other sites
Yah!

I didn't actually try to register vector<vector<*> >, but they seem to have worked for rain dog. I'll dump it into MSVC6 and check it out. I'll also have it register the array type.

Due to the simplicity of overiding native arrays, simply registering the vector with the name 'containedtype[]' should work. No new code required on my part! :D
Though might I ask why you chose to leave erase and insert out by default?

Also, it should be noted that you can't register vector<vector<int> > without first registering vector<int>. It worked for me (No errors from MSVC or AS - but I didn't actually test the run time behavior) Can you show me your code that failed?
[/edit]

[Edited by - Deyja on November 1, 2004 5:00:03 PM]