Sign in to follow this  
RCL

Bring ExecuteStep back :)

Recommended Posts

Hey! Whoa, it was quite a long time since I have checked AngelScript for the last time :) Having downloaded new (2.2.1WIP3) version I was unpleasantly surprised by the fact that ExecuteStep() function was no longer in the API (unless you compiled it with AS_DEPRECATED). That function was just great for scheduling multiple script threads! Consider the example:
int nQuants = GetQuantsForThread( ... );

while ( nQuants-- )
{
    nRet = pThread->m_pCtx->ExecuteStep( asEXEC_STEP_INTO );

    if ( nRet == asEXECUTION_FINISHED )
    {
	// This thread should not be executed anymore
	pThread->CleanUp();
	break;
    }
    else if ( nRet < 0 )
    {
        // handle the errors
    }
}
Now, you say that SetLineCallback() could be used instead of ExecuteStep(). Hmmm... Could you please give me a code snippet which does the same with SetLineCallback()? Cheers, RCL

Share this post


Link to post
Share on other sites
The sample 'events' in the library download shows how the LineCallback can be used to schedule multiple script threads. That scheduler is based on timeouts, i.e. a script is allowed to run for X milliseconds, before it is suspended, but you could easily change this to X steps.

For your convenience, I'm pasting the code here as well:


#include <iostream> // cout
#include <assert.h> // assert()
#include <conio.h> // kbhit(), getch()
#include <windows.h> // timeGetTime()
#include <angelscript.h>
#include "../../../add_on/scriptstring/scriptstring.h"

using namespace std;

// Function prototypes
void ConfigureEngine(asIScriptEngine *engine);
int CompileScript(asIScriptEngine *engine);
void PrintString(string &str);
void LineCallback(asIScriptContext *ctx, DWORD *timeOut);

int main(int argc, char **argv)
{
int r;

// Create the script engine
asIScriptEngine *engine = asCreateScriptEngine(ANGELSCRIPT_VERSION);
if( engine == 0 )
{
cout << "Failed to create script engine." << endl;
return -1;
}

// Configure the script engine with all the functions,
// and variables that the script should be able to use.
ConfigureEngine(engine);

// Compile the script code
r = CompileScript(engine);
if( r < 0 ) return -1;

// Create two contexts, one will run the script thread
// that executes the main function, the other will other
// be used to execute the event functions as needed.
// Note that the global variables declared in the script
// are shared between the contexts, so they are able to
// communicate with each other this way.
asIScriptContext *mainCtx;
r = engine->CreateContext(&mainCtx);
if( r < 0 )
{
cout << "Failed to create the context." << endl;
return -1;
}

asIScriptContext *eventCtx;
r = engine->CreateContext(&eventCtx);
if( r < 0 )
{
cout << "Failed to create the context." << endl;
return -1;
}

// Prepare the script context with the function we wish to execute
r = mainCtx->Prepare(engine->GetFunctionIDByDecl(0, "void main()"));
if( r < 0 )
{
cout << "Failed to prepare the context." << endl;
return -1;
}

// Get the function IDs for the event functions already
int onKeyPressID = engine->GetFunctionIDByDecl(0, "void OnKeyPress()");
int onQuitID = engine->GetFunctionIDByDecl(0, "void OnQuit()");

// Set the line callback so that we can suspend the script execution
// after a certain time. Before executing the script the timeOut variable
// will be set to the time when the script must stop executing. This
// way we will be able to do more than one thing, almost at the same time.
DWORD timeOut;
r = mainCtx->SetLineCallback(asFUNCTION(LineCallback), &timeOut, asCALL_CDECL);
if( r < 0 )
{
cout << "Failed to set the line callback function." << endl;
return -1;
}

// Print some useful information and start the input loop
cout << "Sample event driven script execution using AngelScript " << asGetLibraryVersion() << "." << endl;
cout << "The main script continuosly prints a short string." << endl;
cout << "Press any key to fire an event that will print another string." << endl;
cout << "Press ESC to terminate the application." << endl << endl;

DWORD time = timeGetTime();
DWORD quitTimeOut = 0;

for(;;)
{
// Check if any key was pressed
if( kbhit() )
{
int key = getch();
if( key != 27 )
{
// Fire an event by calling the script function.
eventCtx->Prepare(onKeyPressID);
eventCtx->Execute();

// Note, I'm being a little lazy here, since we don't
// verify the return codes. Neither do I add any safeguard
// against never ending scripts, which if the script is
// badly written could cause the program to hang, I do this
// for the main script though.
}
else
{
// Fire the quit event that will tell the main script to finish
eventCtx->Prepare(onQuitID);
eventCtx->Execute();

// Let the script run for at most 1sec more, to give it time
// to quit graciously. If it does not finish in time the
// script will be aborted.
quitTimeOut = timeGetTime() + 1000;
}
}

// Allow the long running script to execute for 10ms
timeOut = timeGetTime() + 10;
r = mainCtx->Execute();
if( r != asEXECUTION_SUSPENDED )
{
if( quitTimeOut == 0 )
cout << "The script execution finished early." << endl;
break;
}

if( quitTimeOut && quitTimeOut < timeGetTime() )
{
// Abort the long running script.
// AngelScript makes sure everything is correctly freed.
mainCtx->Abort();
break;
}
}

// We must release the contexts when no longer using them
mainCtx->Release();
eventCtx->Release();

// Release the engine
engine->Release();

return 0;
}

void ConfigureEngine(asIScriptEngine *engine)
{
int r;

// Register the script string type
// Look at the implementation for this function for more information
// on how to register a custom string type, and other object types.
// The implementation is in "/add_on/scriptstring/scriptstring.cpp"
RegisterScriptString(engine);

// Register the functions that the scripts will be allowed to use
r = engine->RegisterGlobalFunction("void Print(string &in)", asFUNCTION(PrintString), asCALL_CDECL); assert( r >= 0 );
r = engine->RegisterGlobalFunction("uint GetSystemTime()", asFUNCTION(timeGetTime), asCALL_STDCALL); assert( r >= 0 );
}

class asCOutputStream : public asIOutputStream
{
public:
void Write(const char *text) { buffer += text; }

string buffer;
};

int CompileScript(asIScriptEngine *engine)
{
int r;

// This script prints a char approximately 10 times per second
const char *scriptMain =
"string char = \"-\"; "
"bool doQuit = false; "
"void main() "
"{ "
" uint time = GetSystemTime(); "
" while( !doQuit ) "
" { "
" uint t = GetSystemTime(); "
" if( t - time > 100 ) "
" { "
" time = t; "
" Print(char); "
" } "
" } "
"} ";

const char *scriptEvents =
"void OnKeyPress() "
"{ "
" Print(\"A key was pressed\\n\"); "
" if( char == \"-\" ) "
" char = \"+\"; "
" else "
" char = \"-\"; "
"} "
"void OnQuit() "
"{ "
" doQuit = true; "
"} ";

// Add the script sections that will be compiled into executable code
r = engine->AddScriptSection(0, "scriptMain", scriptMain, strlen(scriptMain), 0, false);
if( r < 0 )
{
cout << "AddScriptSection() failed" << endl;
return -1;
}

r = engine->AddScriptSection(0, "scriptEvents", scriptEvents, strlen(scriptEvents), 0, false);
if( r < 0 )
{
cout << "AddScriptSection() failed" << endl;
return -1;
}

// The script compiler will send any compiler messages to the outstream
asCOutputStream out;

// Compile the script
r = engine->Build(0, &out);
if( r < 0 )
{
cout << "Build() failed" << endl;
return -1;
}

return 0;
}

void PrintString(string &str)
{
cout << str;
}

void LineCallback(asIScriptContext *ctx, DWORD *timeOut)
{
// If the time out is reached we suspend the script
if( *timeOut < timeGetTime() )
ctx->Suspend();
}



I'm sorry about any inconvenience this has caused you, but the ExecuteStep() method was flawed and not nearly as flexible as the new line callback implementation. For the better of the library I decided to remove it.

In the next WIP it will be completely gone (even with AS_DEPRECATED).

Regards,
Andreas

Share this post


Link to post
Share on other sites
The line callback method is really easy to use. I think that it is far far superior to the ExecuteStep() function.

Share this post


Link to post
Share on other sites
Ok, thanks for pointing me out :)

Btw, when I compile AS with warning level set to 4, it produces a lot of warnings. Could you clean up those, please? ;)

Share this post


Link to post
Share on other sites
Hmm... about SetLineCallback superiority...


I see two drawbacks of SLCb() compared to ExecuteStep() (btw, could you clarify why was that flawed?)

1) Code is enlarged (and thus slowed down) with SUSPENDs.

2) SetLineCallback() has a chance of getting called _only_ once per statement, so script execution won't be as smooth as it was with ExecuteLine (because SUSPENDs aren't regularly placed in code).

What I really need is to let thread run for a specified amount of 'time units' (that amount could be small enough, e.g. less than 1 millisecond) and stop running instantly if ran out of its time pool (no matter if it happened after a statement or inside it).

I do not want the thread to Suspend(), I just want it to stop executing for a while so the other threads could have their turn.

Ok, I can use statements as my 'time units', but I think that execution time of different statements varies far more significantly than execution time of opcodes.

Even more, I don't like the absense of actual control of thread execution (what if, for some now-unknown reason, there will be no SUSPENDs in code at all? It is impossible now since the code is compiled by engine, but what if later I decide to allow loading precompiled code, and there will be no SUSPENDs in that?)

So, to sum it up:

1) Could you clarify why ExecuteStep() was so evil and flawed? :)
2) Is there a possibility to control thread execution in a described way? (that is, not depending on good-written code, and with really small 'quants' of time).

Cheers,
RCL

Share this post


Link to post
Share on other sites
The only drawback of the line callback in comparison with the ExecuteStep() is that the callback function needs to be written by the programmer, and then called by the VM for each statement.

By having a line callback function that always suspends, you have the exact same functionality as ExecuteStep(STEP_INTO). I mean, ExecuteStep() also relied on the SUSPEND bytecode, and also always suspended the execution once per statement.

1) This is the basically the same as before. Before the VM could suspend after calling application functions as well, but this was changed to only allow suspension on SUSPEND bytecode for stability.

2) I should think that SUSPENDs are placed regularly enough for you to get smooth execution. Unless your statements call very heavy application functions, you're likely to execute quite a few statements for each millisecond. If your scripts is calling heavy application functions, then you would have the same situation with ExecuteStep() anyway.

You say you don't want the thread to suspend? But that was exactly what ExecuteStep() did when it returned before finishing the script. Nothing has changed in this regard.

AngelScript relies on statements (actually substatements) being treated as units, otherwise you would have big problems keeping everything safe in a multiscript environment. For example, if statements were not treated as units, one script could compute the pointer to an object, be suspended, and another script delete the object, and when the first script resumed the pointer it had already computed would no longer be valid.

Anyway, as I mentioned above, a script statement, is normally quite small. I don't think you will have any problem controlling script execution on a level of milliseconds. Besides the line callback will be able to make it much easier for you, since you don't have to write a loop that continuously calls ExecuteStep() until the time's up. You can now have the line callback check the time and suspend the execution as soon as the time's up.

If you as a programmer allow somebody to write bytecode without SUSPENDs, then you will loose the ability to control execution times. This would be the same for ExecuteStep(). AngelScript relies on the SUSPEND bytecode to know when it is safe to suspend the execution or not, I cannot change this or you would loose stability in your application.

1) It was not so much that it was flawed (perhaps a bad choice of words on my behalf) but more that it wasn't flexible enough, and couldn't do execution on time-outs, or do manually set break points, etc. The line callback can do all that and much more, and still do exactly the same thing as ExecuteStep() did before.

2) No, it is not possible to control thread execution on a smaller time unit. Doing that would compromise stability and safety as described above. If you want control on that low a level you should probably be using OS threads instead of script threads, and you would still have to deal with all the added complexity of a multithreaded environment.

I can assure you, that you didn't loose anything with the change to line callbacks. The current version has slightly less performance than previous versions, but this is not because of the line callbacks. The decrease in performance is because the added overall stability and safety in the VM. I have every intention of bringing the performance back up to previous levels, and even beyond. But you'll have to be a little patient with this as there is a lot of other things I also want to implement.

Regards,
Andreas

Share this post


Link to post
Share on other sites
Quote:
Original post by WitchLord
AngelScript relies on statements (actually substatements) being treated as units, otherwise you would have big problems keeping everything safe in a multiscript environment. For example, if statements were not treated as units, one script could compute the pointer to an object, be suspended, and another script delete the object, and when the first script resumed the pointer it had already computed would no longer be valid.


By the way, can't it happen now (between the statements)?

Quote:
Original post by WitchLord
If you as a programmer allow somebody to write bytecode without SUSPENDs, then you will loose the ability to control execution times. This would be the same for ExecuteStep(). AngelScript relies on the SUSPEND bytecode to know when it is safe to suspend the execution or not, I cannot change this or you would loose stability in your application.

[...skip...]

2) No, it is not possible to control thread execution on a smaller time unit. Doing that would compromise stability and safety as described above. If you want control on that low a level you should probably be using OS threads instead of script threads, and you would still have to deal with all the added complexity of a multithreaded environment.


Well, OS threads are something I don't want to resort to, because they introduce a lot more other problems.

Actually, it's a pity that AngelScript can't be used as a some kind of 'virtual processor'. That's probably the closest word to describe functionality I want ;)

Cheers,
RCL

P.S. What about cleaning up compiler warnings (/Wall or /W4 in cl)? They aren't critical, but it's better not to have those at all. Some ppl might have /WX (threat warnings as errors) option turned on ;)

Share this post


Link to post
Share on other sites
I've worked hard to make AngelScript as safe as possible. This among other things, means that it at all times guarantee the validity of any pointers. So, unless I've done something wrong, you won't be able to get invalid pointers between statements. This of course assumes that the application cooperates.

I agree that OS threads bring a lot of problems that have to be carefully weighed against any advantages they may bring.

Quote:

Actually, it's a pity that AngelScript can't be used as a some kind of 'virtual processor'. That's probably the closest word to describe functionality I want ;)


What is missing in AngelScript that you need? AngelScript can already run multiple scripts in parallel, and you can do either fully automatic scheduling of the script threads (by using the line callback), or allow the scripts to do their own manual scheduling (by registering methods to sleep, wait, etc).

I'll look into clearing up the warnings.

Share this post


Link to post
Share on other sites
Quote:
Original post by WitchLord
Quote:

Actually, it's a pity that AngelScript can't be used as a some kind of 'virtual processor'. That's probably the closest word to describe functionality I want ;)


What is missing in AngelScript that you need? AngelScript can already run multiple scripts in parallel, and you can do either fully automatic scheduling of the script threads (by using the line callback), or allow the scripts to do their own manual scheduling (by registering methods to sleep, wait, etc).


Well, I agree that is enough for most applications. And can be used in my case, too :)

I just wanted the (inherently unsafe) ability to execute script opcode-by-opcode, with the possibility to stop at any moment, no matter if the script code 'allows' it or not.

Well, if you are not against that, I'll try to add such function to AS myself (just for my own internal usage) ;)

Cheers,
RCL

Share this post


Link to post
Share on other sites
I understand.

Well, it's not something that I will officially add to the library. But it shouldn't be difficult for you to change the Execute() method to only execute one bytecode and then return.

Share this post


Link to post
Share on other sites
Why would you want to execute the script opcode by opcode? Sure you can do the same thing with an x86 debugger and native code, but why would you wish for the ability here?

I would think it to be a giant waste to be writing your scripts in the as bytecode and that could be the only reason I came up with to require the need to debug bytecode at a time.

Share this post


Link to post
Share on other sites
I thought he was trying to even out time-step sizes when threading scripts. It's inherently difficult because you can't pause in the middle of application functions. There also seems to be a misunderstanding about the suspend stuff. I thought the compiler stuck them in automatically.

Share this post


Link to post
Share on other sites
Quote:
Original post by Rain Dog
Why would you want to execute the script opcode by opcode? Sure you can do the same thing with an x86 debugger and native code, but why would you wish for the ability here?

I would think it to be a giant waste to be writing your scripts in the as bytecode and that could be the only reason I came up with to require the need to debug bytecode at a time.


I don't need that function for debugging purposes. I want to run script threads concurrently with the ability to preempt one thread by another when the scheduler wants, not when the code allows :)

It actually does not matter, whether an atomic execution unit is opcode or anything else, as far as following conditions are satisfied:

1) execution time of an atomic unit is small enough
2) that time is more or less constant
3) thread can be stopped after executing arbitrary atomic unit

Well, SetLineCallback()/ExecuteStep() almost satisfy those, except for the 3), because thread can be only stopped when SUSPEND occurs in the code, and there can be no SUSPENDs in the code at all.

I think I will customize AS a bit, adding that unsafe ExecuteUnit() function, and define BUILD_WITHOUT_LINE_CUES, so script code will contain no SUSPENDs (and thus will be smaller and faster).

Cheers,
RCL

P.S. ASM VM seems to be broken in current release? At least some assert()s that check for offsets do not compile with USE_AS_VM.

Share this post


Link to post
Share on other sites
The only reason why there would be no SUSPENDs is if someone tampers with your precompiled bytecode (for example by building a customized compiler with the BUILD_WITHOUT_LINE_CUES). You should make sure to add validations that will tell you if the code was produced by your own compiler and not by someone elses. Assuming that you're going to use precompiled bytecode at all, because if not then it will be very difficult for anyone to remove the SUSPENDs (only by hacking the application).

Yes, the ASM VM is temporarily out of order (as noted in the change log). I'll fix it as I release the final version of the library. In either case you won't be able to use the ASM VM for your ExecuteUnit(), since the ASM VM doesn't return for each bytecode.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this