15 replies to this topic

[Ameise]

I need to execute Angelscript bytecode in my own Virtual Machine as I need to perform tasks as part of the execution that the AngelScript VM is incapable of. Currently, I am using SaveByteCode on the module, and then trying to parse the resultant bytecode. The problem so far is that I am unsure what the format of this bytecode is... it's not straight bytecodes going straight in.

I wrote a very simple AS script:

 

void run ()
{
	print("RUN CALLED");
}

 

 

Which results in the following output:

 

01 00 00 00 00 00 01 66 6E 03 		? ? ? ? ? ? ? f n ? 
72 75 6E 40 4E 00 00 00 00 01 		r u n @ N ? ? ? ? ? 
00 00 00 0C 3F 3C 00 3D 00 3B 		? ? ? ? ? < ? = ? ; 
04 01 3D 01 04 01 3D 02 04 01 		? ? = ? ? ? = ? ? ? 
3D 03 3F 0A 00 01 01 6F 6E 06 		= ? ? ? ? ? ? o n ? 
73 74 72 69 6E 67 00 04 01 00 		s t r i n g ? ? ? ? 
02 06 01 01 0A 01 00 00 00 01 		? ? ? ? ? ? ? ? ? ? 
72 00 00 00 00 05 61 6E 10 5F 		r ? ? ? ? ? a n ? _ 
73 74 72 69 6E 67 5F 66 61 63 		s t r i n g _ f a c 
74 6F 72 79 5F 05 6F 72 01 00 		t o r y _ ? o r ? ? 
00 00 01 01 02 40 42 00 01 40 		? ? ? ? ? @ B ? ? @ 
4A 01 01 00 00 00 00 00 61 6E 		J ? ? ? ? ? ? ? a n 
07 5F 62 65 68 5F 30 5F 00 00 		? _ b e h _ 0 _ ? ? 
01 00 01 01 01 00 00 6F 72 01 		? ? ? ? ? ? ? o r ? 
00 00 61 6E 05 70 72 69 6E 74 		? ? a n ? p r i n t 
00 00 01 00 01 01 01 00 00 00 		? ? ? ? ? ? ? ? ? ? 
00 61 6E 07 5F 62 65 68 5F 31 		? a n ? _ b e h _ 1 
5F 00 00 00 00 00 00 6F 72 01 		_ ? ? ? ? ? ? o r ? 
00 00 6E 00 01 6E 0A 52 55 4E 		? ? n ? ? n ? R U N 
20 43 41 4C 4C 45 44 00 00 00 		  C A L L E D ? ? ? 

 

 

How would I go about loading this and other scripts? The AngelScript documentation doesn't go into detail about the specifics of precompiled bytecode, only the API side of things.

Edited by Ameise, 15 February 2013 - 12:34 AM.

[Andreas Jonsson]

The saved bytecode is not documented as it was never meant to be interpreted outside the AngelScript library. The best you can do is to take a look at how asCReader in as_restore.cpp reads the saved bytecode. However, I make no promise to keep this same format with upcoming releases.

 

What exactly is it that the AngelScript VM isn't capable of that you need to execute in your own VM? 

[Ameise]

I need the ability to both track how many bytecode ops were executed during a pass, and also the ability to terminate execution after surpassing a certain number of operations, and the ability to resume it later. I did not see any such functionality during a cursory overview of the API. I'm not really interested in speed or performance, only guaranteeing that I can limit the number of operations executed per 'tick'. Ideally, function calls and operations directly related to the function call (such as pushing arguments) wouldn't be counted towards the op-limit (to make up for the lack of inlining).

[Andreas Jonsson]

You should ideally be using the line callback for this. See Manual: Timing out long running scripts

 

A script execution that has been suspended with a call to asIScriptContext::Suspend() can be resumed by calling asIScriptContext::Execute().

[Ameise]

You should ideally be using the line callback for this. See Manual: Timing out long running scripts

 

A script execution that has been suspended with a call to asIScriptContext::Suspend() can be resumed by calling asIScriptContext::Execute().

 

In the sense that I mean, for the purposes of the project, this behavior should be entirely transparent to the end-user writing the scripts, and I need to limit/track based upon actual bytecode operations themselves, and not per line. I require bytecode-level granularity. I require it to be both deterministic (which suspending using time isn't) and for other reasons, I need the actual bytecode count and to be able to limit it therewith.

Edited by Ameise, 15 February 2013 - 10:43 AM.

[Andreas Jonsson]

OK. In that case I suggest you customize asCContext itself to do what you need.

 

It will most likely be easier to do so than to try to interpret the bytecode on your own. It will also be much easier for you to maintain your customized version this way with upcoming changes to the library.

 

The interpretation of the bytecode is done in asCContext::ExecuteNext(). You can easily change to count the number of executed instructions and break out of the loop whenever you need.

 

Regards,

Andreas

[Ameise]

OK. In that case I suggest you customize asCContext itself to do what you need.

 

It will most likely be easier to do so than to try to interpret the bytecode on your own. It will also be much easier for you to maintain your customized version this way with upcoming changes to the library.

 

The interpretation of the bytecode is done in asCContext::ExecuteNext(). You can easily change to count the number of executed instructions and break out of the loop whenever you need.

 

Regards,

Andreas

Excellent.

Also, is there any hint as to when function inlining might be implemented? For my purposes, it would be best if function calls aren't tracked at all, and inlining would be perfect for that. Lacking that, would there be a good way to 'skip' function calls and dependent push/pops for op-counting?

[Andreas Jonsson]

It will likely take quite a while before I get to implementing function inlining. 

 

As you'll customize the VM you can quite easily skip counting the asBC_CALLxxx instructions and asBC_RET. 

 

I'm curious about what it is that you're working on that you need such fine control over the execution. Can you elaborate on it?

[Ameise]

It will likely take quite a while before I get to implementing function inlining. 

 

As you'll customize the VM you can quite easily skip counting the asBC_CALLxxx instructions and asBC_RET. 

 

I'm curious about what it is that you're working on that you need such fine control over the execution. Can you elaborate on it?

 

Certainly. My goal is to do something similar to MIT's battlecode: http://www.battlecode.org/

The basic premise is that you have two competing AI's. There are several guarantees/restrictions - the simulation is deterministic (that is, each time you run it you get the same results), the simulation is fair (that is, each turn you have to give each bot the same number of operations, as raw time is not guaranteed), and the simulation is challenging (you not only count bytecode ops towards the per-turn limit, but also certain API calls have a specific cost as well).

My goal was to make up for some of the deficiencies I perceived in this year's battlecode:
 

1. The competition as it is, due to using Java (nothing against Java in this situation), discourages proper programming techniques. Mainly, javac doesn't really optimize Java code (as that's the job of the HotSpot JIT), and since they are interpreting it, things such as function/method calls cost the programmer, so their code usually ends up as an illegible block - you are virtually penalized for writing your code well; you should never have you assume that the compiler is inept.
2. They appear to have simplified the competition drastically so that 'Zerg Rush' tactics are the only real solution.

For the before-stated reasons, though, the number of operations need to be counted and I need it to return when we are going to reach the limit on the next turn, to make sure it's fair - each bot gets the same timeslice, op-wise. I had four languages I was considering for this: AngelScript, Squirrel, NullC, and C#. I've already ruled out Squirrel, and also C# as forcing Mono to behave in this fashion would likely be an insurmountable task.

I do wonder - are there any good editors for AngelScript akin to Eclipse for Java?

Edited by Ameise, 15 February 2013 - 06:12 PM.

[Andreas Jonsson]

Now your questions make perfect sense to me. 

 

It sounds like a fun project. I look forward to seeing more of it.

 

I doubt there are any stand-alone IDEs for AngelScript, as the scripting library is for embedding into application rather than something to create standalone programs with. For this reason the IDE would have to understand the application in which the scripts will be executed in order to be useful. Most users tend to write their own IDEs as part of their content creation tools.

 

If you're only looking for source editor with syntax highlighting though I'd recommend Notepad++, or Scite. You can visualize the AngelScript code as if it is C++ and the syntax highlighter does a pretty good job. 

[Ameise]

Now your questions make perfect sense to me. 

 

It sounds like a fun project. I look forward to seeing more of it.

 

I doubt there are any stand-alone IDEs for AngelScript, as the scripting library is for embedding into application rather than something to create standalone programs with. For this reason the IDE would have to understand the application in which the scripts will be executed in order to be useful. Most users tend to write their own IDEs as part of their content creation tools.

 

If you're only looking for source editor with syntax highlighting though I'd recommend Notepad++, or Scite. You can visualize the AngelScript code as if it is C++ and the syntax highlighter does a pretty good job. 

If I get iterative execution (I really have no idea what to call it) working well enough, I'll send you a patch. It shouldn't be too difficult. Even if you don't need to know the opcount of the execution, there are other situations where you want to limit it without having to change the script itself.

[Tzarls]

Take a look at this thread:

 

http://www.gamedev.net/topic/636508-angelscript-on-flascc/#entry5015807

 

The guy there talks about a code editor he´s made... maybe you can get in touch with him and see if he can share his sources?

[Ameise]

Take a look at this thread:

 

http://www.gamedev.net/topic/636508-angelscript-on-flascc/#entry5015807

 

The guy there talks about a code editor he´s made... maybe you can get in touch with him and see if he can share his sources?

I'll have to take a look at that, thanks.

Also, Andreas, I've successfully made it so it can be interrupted. What I've done is somewhat hackish so I want to clean it up first. I was thrown off by the copying of the register data in the start of ExecuteNext, as there is apparently local data in l_** which is not in the registers, so it was being blown away when I returned within the loop. I solved that with some struct/RAII trickery.

On the original premise of this post, though - is it possible to get some sort of assembly readout, even if text? Something users can look at and go "Oh, that's where my opcodes went!"?

Edited by Ameise, 16 February 2013 - 01:12 AM.

[TheAtom]

About the IDE: if you can make your (postprocessed) AS code appear as close to C++ as possible (enforce semicolons at the end of class definition, use array<int> instead of int[] etc.), #include a file that defines some keywords (like #define interface class), and autogenerate "headers" with definitions of your registered classes and functions, then Visual Studios (including the free Express editions) prior to 2010 do more than good job for working with AS. Syntax highlight, type sensitive (sic) autocomplete, jumping to code definition, search for symbol references, class view and all other features work without any problems. The only thing that this setup is lacking is the embedded debugger.

 

The (detailed) propaganda continues here: http://www.gamedev.net/topic/594639-symbols-table-andor-syntax-tree/#entry4806355

with screens: http://www.gamedev.net/topic/603001-creating-as-ide-thoughts/#entry4817006

 

It requires some initial, one-time investment of time to make it work, but it's definitely worth it. And let me stress it again that it works with free versions of MSVC,

Edited by TheAtom, 16 February 2013 - 09:15 AM.

[Andreas Jonsson]

Take a look at this thread:

 

http://www.gamedev.net/topic/636508-angelscript-on-flascc/#entry5015807

 

The guy there talks about a code editor he´s made... maybe you can get in touch with him and see if he can share his sources?

He posted the link to the source code further down in that thread. He also said that the IDE is tied into his engine, which is exactly what I mentioned earlier. Still, the code for his IDE might be a good start to write your own if you intend to do that.

 

Also, Andreas, I've successfully made it so it can be interrupted. What I've done is somewhat hackish so I want to clean it up first. I was thrown off by the copying of the register data in the start of ExecuteNext, as there is apparently local data in l_** which is not in the registers, so it was being blown away when I returned within the loop. I solved that with some struct/RAII trickery.

On the original premise of this post, though - is it possible to get some sort of assembly readout, even if text? Something users can look at and go "Oh, that's where my opcodes went!"?

 

The local variables are for optimizations. The compiler is able to put the local variable into the CPU registers, but the context's class members can't be placed in the CPU registers, so using those directly is a lot slower.

 

I have an output of the bytecode in readable text for debugging purposes. You can probably use that one in your project. It's in asCByteCode::DebugOutput().

[Tzarls]

He posted the link to the source code further down in that thread. He also said that the IDE is tied into his engine, which is exactly what I mentioned earlier. Still, the code for his IDE might be a good start to write your own if you intend to do that.

 

Sorry.... somehow I got the idea he was releasing only the binaries as his IDE is tied to the game...