Archived

This topic is now archived and is closed to further replies.

Giving Some Back

This topic is 4997 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hey all, I just completed what I consider to be a very large feat, and I would like to help give back to the GDNet isometric community by offering advice on the subject and perhaps some source code too, I''ll explain. I''ve just completed a scripting engine, which is being put into my game engine, it uses a semi-modified version of C as it''s language, it compiles down to an instruction based hierarcial object model. While building this I had to overcome some _very_ hard obstacles and I would like to offer my experience and solutions to those who would undertake such a project by fielding questions. This may seem a bit out of place on the Isometric forum, but this forum is really where ''my'' community is and since alot of you know of me it may seem a little more relavant here than somewhere else. Anyway, feel free to post questions here and I will answer them to the best of my ability. Happy Coding, Raymond Jacobs, www.EDIGames.com www.EtherealDarkness.com

Share this post


Link to post
Share on other sites
think you could write up a tutorial on your methods of making it? Maybe without being language specific, or you could kinda'' do both. I''m using VB and still learning c++ and I''ve always wondered how I could make one myself. Things that would be useful would be like the techniques used to accomplish the basics such as function calling, loops, variables and mathamatics, etc.

Share this post


Link to post
Share on other sites
JoHnOnIzEr:
I would really like to do that, but at the moment with my current development schedule I dont have the time (job+game=16 hour days).

Herr_O:
the main reason is we wanted C as the scripting language, also I wanted total control over specifics, like, weak or strong typing and variable scope. It is also notable our scripting languge needed somthing we call ''event locking'' that is when the script makes a call to the engine the engine can lock the execution of the script with the contingency of a certain event, when the event is fired execution resumes, i wasnt sure if lua could do this, but again i really didint check.

- could we have used lua?

Probably, but I saw no reason to be dependant on a 3rd party if i didint have to.

e.g.
we need a new language feature, lua cant do it, we now not only have to implement this new feature ourselvs, but construct a whole new scripting engine which is compatible with all the scripts we have written.

while this is probably rare, i''d rather not introduce any problems=)



-was it worth it to write our own scripting engine?

considering it only took me a week to write the Parser,Compiler and Virtual Machine, definetly, the time to gain ratio in this case was excellent. I would imagine though, that most people couldnt write one in such a short period of time, even when time isnt considerd it is a daunting task, and you can easily travel the 90 meters and trip on the last 10.

Raymond Jacobs,

www.EDIGames.com

www.EtherealDarkness.com

Share this post


Link to post
Share on other sites
objects!!!
I want to know how do you handle objects!
and if you can show the bitmaps of some objects, how do you know what tiles each object will block and that stuff...
my RTS game is allmost lost because I dont know how to handle objects in game properly
if you can post some of your code, or explain how do you do it, you will make me an happy person :-D
I saw your game, I played a version you released, very nice
TIA

Share this post


Link to post
Share on other sites
Hmm ok. I''m pretty new to scripting and I''m still tied up with my editor so I haven''t gotten around seeing what’s possible. Thanks for the reply! =)

-----------------------------


Share this post


Link to post
Share on other sites
I''d be interested in the techniques that you used to implement the "locking" feature. I''m currently designing the scripting language for my game and would undoubtedly find the information useful.

I need to be able to:

1) run very small scripts in whole while the game engine waits for them to complete before moving on.

2) execute the global body of a script during the load, and then call part of a function from it each frame based on time slices or priority levels.

3) detect when a function or script is completed so that it can be released.

4) compile and update a list of scripts that will be executed each frame using time slices or priorities.

5) pause execution of a script at any given time and then continue with the script as the need arises.

These issues seem like the same thing that you were having trouble with, so any information that you can provide in the regaurd would be literally invaluable to be.

Share this post


Link to post
Share on other sites
Ok,

a little offtopic, but i feel it''s neccisary that you know my mood, since my post might be a little brief.

Our writter just quit on us, so I am scrambling to find a replacement so that we can hopefully still meet our game deadline.

ok with that said, I will answer your questions ChronosVagari.

>>1) run very small scripts in whole while the game engine waits >>for them to complete before moving on.

this is very simple, when you execute your script it should act as if you are calling a function. meaning as the script executes the ''RunScript'' function doesnt return until the script is done executing, this is simple to acomplish since when running the script you simply have it run in this fashion.

//Psuedo Code
RunScript(script)
{
//set the instruction pointer to the first instruction in the script
m_lpexe=script.firstinstruction;
//Resume/begin Execution of the script
ResumeExecution
}

ResumeExecution()
{
//run to infinity
while(1)
{
//execute the next instruction
//if it comes back as 0 it means the script has ended
//or somthing else has happened to halt script execution
//ResumeExecution can be called again to resume the execution
if(!ExecuteNextInstruction())break;
}
}

since this is done in a while, the RunScript call will not return until the script is done executing.



>>2) execute the global body of a script during the load, and >>then call part of a function from it each frame based on time >>slices or priority levels.

not sure im entirely clear on this,but here is how i would do it.

your script can have ''entry points'' that is functions in the script that can be called externaly via a RunScript command of internaly depending on how your language calls functions.

you can use a predefined ''Main'' entry point that your engine can designate to be called when somthing ''loads'', you can then call other entry points as you wish, for instance ''EnterFrame'' for each frame.

if you plan to execute the script every frame is is important that your scriptinglanguage be compiled down (on first run) to an object model, instead of parsing it each time, the object model should be saved using the file''s name and path so that when the script is meant to be run again it can use the stored model instead of reparsing the script.




>>3) detect when a function or script is completed so that it >>can be released.

well in consideration of your above posts you would not ever want to release a script, until the game has ended, though if you really want it to be tidy you can designate when loading a script that it should not be stored, in such a case when the script is finished it''s execution (ResumeExecution reutrns) you can free the script.


>>4) compile and update a list of scripts that will be executed >>each frame using time slices or priorities.

you could easily create a function called... ''AddScript'' to compile and add the script model pointer to an array attached to the frame, in accordance with the other rules the script could be designated as persistant or volatile. This list of scripts can be executed one by one using a for loop and the RunScript command.



>>5) pause execution of a script at any given time and then >>continue with the script as the need arises.

Pausing the execution of a script is easy, if the script is built right. Your object model should be instruction based, an instruction being things like, IF,Switch,Case,Else,Break,While,Call,Return etc.

your script should be able to be executed with only the knowledge of one instruction.. ill explain...

Each instruction should have a next and prev pointer, pointing to it''s next and previous siblings in the hierarcial model. it should also have a parent pointer, an example of this is, the instructions inside an if statement would have if as it''s parent.
it should also have a pointer to it''s first child.
if you need more clarification on this model i can illustrate it.

at any point in your instruction executor you can return 0, doing so will stop the execution in it''s tracks, be sure to queue up the next instruction first though, so that when execution begins it will be ready to execute the next instruction. for stoping execution via real code you can have a variable placed as the condition of the ResumeExecution while when, lets say, m_executing is false the while breaks and execution halts, calling ResumeExecution should make this variable true again, and resume execution.

Hope that answers your questions.


Raymond Jacobs,

www.EDIGames.com

www.EtherealDarkness.com

Share this post


Link to post
Share on other sites
Hey, I just wonderd how you handle stuff like classes
and such ?

Also do you use a stack based aproch on the memory or a register based one ? or maybe something else that i havent heard of :D

Share this post


Link to post
Share on other sites
the scripting language i wrote is C basicly.

so no classes=)


as for memory, i use somthing i designed sortof.

for each instruction there is a list of associated data segments each dataseg is either a string or a double, for instance an if might be.

[1][&&][2][||][3][>][5]

each dataseg has a usage flag too, it can be either
UTVoid - a typeless data segment
UTConstant - a constant number or string
UTVariable - a string name coresponding to a variable

in the event a UTVariable dataseg is encounterd it is resolved by getting the variable value(which is also a dataseg) from the script executors variable array (all variables are global within the scope of the executor object(the game in this case))if the variable didint exist(this is allowed due to typeless and implicit variables)then a new void dataseg is created for the variable name and stored,if it did exist then the real value is returned as it''s dataseg pointer and can be used in the computation as if it were a constant(since all variable''s datasegs are constants anyway) in this sense the only way to use a variable in the language is by reference, this make is possible to have a script call to an engine function allow the engine function handler to not only specify a return value, but also to modify the incomming arguments *only if they are variables of course*



Raymond Jacobs,

www.EDIGames.com

www.EtherealDarkness.com

Share this post


Link to post
Share on other sites
Cool. Thanx for taking the time out to help a fellow dev''r. I was thinking about this stuff pretty much all day inbetween customers at work and I think that I''ve got a lot of it sorted out now. Your comments definitely help, but I''ll have to re-read them later. Right now my brain is fried from too much work and not enough food or water. However, you''ve already filled in some of the hazey aspects. I''m also sure that more of it will become aparent to me after I have some time to chill. I''m sure that I''ll want to rap about this with you more after a while.

...don''t worry if you can''t respond to furthur questions/comments in a timely matter. I know that you''ve got a lot going on right now. I''m a one man team doing game deving on my free time. Though it''s a serious hobby for me, it''s just that...a hobby.

Later gator.

Share this post


Link to post
Share on other sites
Thanx again bro! These were some particularly helpful points.

quote:
Original post by EDI
[...]
you can use a predefined ''Main'' entry point that your engine can designate to be called when somthing ''loads'', you can then call other entry points as you wish, for instance ''EnterFrame'' for each frame.
[...]
well in consideration of your above posts you would not ever want to release a script, until the game has ended, though if you really want it to be tidy you can designate when loading a script that it should not be stored, in such a case when the script is finished it''s execution (ResumeExecution reutrns) you can free the script.
[...]
you could easily create a function called... ''AddScript'' to compile and add the script model pointer to an array attached to the frame, in accordance with the other rules the script could be designated as persistant or volatile. This list of scripts can be executed one by one using a for loop and the RunScript command.
[...]
at any point in your instruction executor you can return 0, doing so will stop the execution in it''s tracks, be sure to queue up the next instruction first though, so that when execution begins it will be ready to execute the next instruction. for stoping execution via real code you can have a variable placed as the condition of the ResumeExecution while when, lets say, m_executing is false the while breaks and execution halts, calling ResumeExecution should make this variable true again, and resume execution.
[...]


I''ve got a pretty good handle on the concepts so far. However, this is the first scripting language that I''ve implemented (besides Lua and Python) thus far and it turned out to be more complex than I had previously thought. I still have a few more points to research before I move on to implementation, but you''ve actually cleared up some things that were bugging the hell out of me. Anyway, I''ll let you know if I need any more insight.

Share this post


Link to post
Share on other sites
At what point in the script's lifespan is this a diagram of (High-Level Code, Low-Level Code, Compiled Code, or Virtual Machine)? I'm having a little trouble following it. Are you storing fuction info in your variable heap as well?

Maybe if you laid out the script's lifespan using text then I'd be able to follow it a little better.



I just finished designing and implementing my Low-Level Code and Compilier. Right now I'm working on the Virtual Machine that actually runs the script. This seems to be the hardest part for me so far.

***************************************************
;here's an example of my low-level code:

; GLOBALS
VAR G_ONE
VAR G_TWO

FUNC ADDVARS
{
; POP PARAMETERS FROM THE STACK
PARAM ONE
PARAM TWO
; DECLARE A TYPELESS VARIABLE IN LOCAL SPACE
VAR TEMP
; ONE + TWO
MOV TEMP, ONE
ADD TEMP, TWO
; MOVE THE VALUE INTO THE RETURN REGISTER
MOV _RETVAL, TEMP
}

; _MAIN IS AUTOMATICALLY EXECUTED WHEN THE SCRIPT IS LOADED
FUNC _MAIN
{
VAR RESULT
MOV G_ONE, 1
MOV G_TWO, 2
; PUSH THE PARAMETERS ONTO THE STACK
PUSH G_ONE
PUSH G_TWO
; CALL MY FUNCTION
CALL ADDVARS
; RETRIEVE THE RESULT, _RETVAL NOW HAS THE VALUE OF 3
MOV RESULT, _RETVAL
}
***************************************************************

Then I do a two-pass on the code, building all of my tables (functions, numerical variables, constants, string, jumps, etc.) on the first pass, and then compiling down to bytecode on the second using the reference tables.


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~ The only thing better than word-play is playing with words involving word-play. ~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


[edited by - ChronosVagari on April 1, 2004 4:39:51 PM]

Share this post


Link to post
Share on other sites
Glad to see it's coming along.


one difference between your scripting language and mine is, the ommision of low-level code.

i compile direct from high-level C(esc) to bytecode, i didnt see a point to having a middle-man.

as for my scripting engine there arnt and script definable functions, all function calls get passed to the engine to be handled, this keeps the scripting language role very
'command based'

when you would call a function it is stored as an instruction.

with a bunch of datasegs
IT_CALL[ret,arg,...]

as for the diagram, it really shows the main objects in my scripting engine.

you have the instruction object which is what the bytecode model is made out of,(that and datasegs).

there is then the script object, which does the job of compiling a script down to bytecode,*or openening a bytecode script*

then you 'plug' a script object into an executor object, to execute the script.

in my engine an executor object is anything extending FC_Base,

so the game object FC_2DFlare, can execute scripts, since when you call a function via script, it calls a member function (HandleScriptCall), so the actual functionality the script allows is based on the executor.

i.e.
The game object may be able to handle ret=GetVideoAdaptor();

but the sprite object couldnt, but could handle, GetCurrentDirection();

as for the executor(VM) being the hardest part i would have to agree, mine was pretty complicated due to the fact i had to write an infix interpreter, looks like your code is more twoards ASM, which might be easier for you.

the key is the bytecode model, using the doubly linked list model for instructions allows orientation forward,backward,tochild and toparent, this is key for being able ot jump around in your code, when all you have as a reference is a single instruction.
Good Luck =D


Raymond Jacobs,

www.EDIGames.com

www.EtherealDarkness.com



[edited by - EDI on April 1, 2004 5:44:19 PM]

Share this post


Link to post
Share on other sites
When I set off to make my scripting language I ''felt'' that i knew enough to do it, as for external resources I didint use any, so your guess is as good as mine, all i can say is be fimilar and comfortable with the knowledge of how code executes.

the major pitfalls that got me, were really circumstantial for my specific case *event locking*, though i would now live and die by these few rules....


if your planing to use infix expressions, save yourself some time and find some infix evaluator code online.

ability to traverse the final instruction code with knowledge of only one instruction, is a must, that is from a given instruction you must be able to go to that instructions nested parent,it''s first child, it''s previous sibling and it''s next sibling.

as mentioned above instruction execution should happen on a per instruction basis, the ExecuteInstruction function should make use of, and move an instruction pointer, based on the kind of instructions that get executed.

e.g.

if you execue a variable set command, then advance the isntruction to it''s next sibbling(if applicable)

if you execute an if command, evaluate the infix terms if true advance to the instruction''s first child(if applicable), otherwise advance to the next sibbling(if applicable).

and as with all things, obey the rules of K.I.S.S =D




Raymond Jacobs,

www.EDIGames.com

www.EtherealDarkness.com

Share this post


Link to post
Share on other sites
quote:
Original post by Nerothos
I recently started my own scripting language. Do you have any good resources or ideas / major problems you encountered along the way that I could use as a base step? Thanks a lot!


If you are going to make a Command Based Language (like EDI has) wherein your game functions are written in C++ and are called from your compiled scripts via calls to those functions (commands), then you may be able to get by with what you already know. Just attack the job in small pieces, take your time, and pay attention to details. You'll do fine.

If you're making a full fledged Procedural Language (like I am) which is much more similar to something like straight C, then you're in for a rough ride. This is, without a doubt, the toughest programming challenge that I've set out to do. However, the applications of such a system will prove to be well worth it in my projects. Well designed Procedural Languages can do everything that Command Based Languages can do and then some with little or no run-time loss, are extremely flexible, and are 100% reusable.

But, on flip side, they take WAY longer to implement and will give you a lot of headaches along the way. I'm about half-way through the process and have been hacking away at it for 2 months. And, while I have no doubt what-so-ever that I will finish this task (I always do), It will likely take me 2 more months to complete...whereas EDI's Command Based Language took roughly 1-2 weeks to implement.

If you are serious about designing a scripting language then I HIGHLY suggest a new book called "Game Scripting Mastery" by Alex Varanese, especially if you're making a Procedural Language. Procedural Languages go into some pretty heavy Compiler Theory and should NOT be attacked without devoting a hefty amount of in-depth research and design time.

Also remember, as with any scripting system, try to implement as much is practically possible in straight C++ (for raw speed), and only use scripts where it makes sense to because of the developmental advantages.

[edited by - ChronosVagari on April 1, 2004 9:38:33 PM]

Share this post


Link to post
Share on other sites
I would just like to clarify,

the only thing that the scripting language i wrote lacks, is the ability to define and call script based functions.

the only reason i didint implement this was because there was really no need for it, and to have them, would cause the users of the scripting language to use it in ways that it should not be used *writing lots of code in script wherein it should belong in the engine*

with that said, chances are my language is about 20% away from where CV's will be when it's done, since implementing functions is not that hard. in fact it would have been one of the easier challenges that I faced. to give a litte orientation of language capability here is most the functionality my scripting language can handle.

#an entry point
Main()
{
x=25;
if(x>30)
{
bln=false;
}
else
{
bln=true;
if(x==25&&bln==true)
{
v=0;
loop=true;
while(loop)
{
switch(v)
{
case 0:
v=v+1;
break;
case 1:
v="cool";
break;
case "cool":
v="wow";
break;
default:
loop=false;
break;
}
}

#call an internal function
yn=DoesEDIGamesRock();
if(yn==true)
{
#end the script
return;
}
else
{
#unreachable statement ;-)
}
}
}
}


Raymond Jacobs,

www.EDIGames.com

www.EtherealDarkness.com



[edited by - EDI on April 1, 2004 9:42:27 PM]

Share this post


Link to post
Share on other sites
Just to reiterate EDI''s above post, the only difference that I see in the actual script code is the comments. Mine uses the standard C // and /* */ syntax. Other than that, that code example would be exactly the same in my language.

But, for me, the ability to call functions defined within a script from other scripts as well as the host API is well worth the extra effort...plus, as a hobbiest, I like the challenge.

Share this post


Link to post
Share on other sites