• Advertisement

Archived

This topic is now archived and is closed to further replies.

Creating A Scripting Engine

This topic is 5500 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I''m trying to create a complex scripting engine run by a VM, but I''m having a hard time resolving a few issues: First of all, all the tutorial''s I''ve read use a stack machine and one data type, usually integer. I plan on having integers, a floating point type, and a string type. Should I have a different stack for each type, or would it be more efficient to have a single stack of a base object type and have the variable types derived from that so they could all be pushed and popped on a single stack? It seems like the first method would be best, but as I''ve never done it before I''m not 100% sure. Secondly: I plan on allowing custom types in the scripting language. What is the best way to handle those? I''m not really sure other than possibly treating it as a collection of individual varibles, but pushing 50 different variables on the stack to send to a function (for ex) seems messy and inefficient. Also, it would be nice to be able to pass one of the objects created in script to a game function. Third) What would be the best way to implement arrays in the scripting language? If I go with multiple stacks from #1 should I have a second stack for each type that holds arrays of that type? I want arrays to be dynamically sized and I''m not real sure how I should keep track of that. Fourth, what is the best way to handle function calls? Pushing variables onto the stack wouldn''t work like it does in x86 asm unless I implement a lot of modifiers to be able to access variables from anywhere on the stack instead of just pushing/popping. Is there any easier way that isn''t too slow? I''ve already read most of the tutorials I could find on the internet about scripting languages, including ones here at gamedev.net, on flipcode.com, and at gamasutra.com.

Share this post


Link to post
Share on other sites
Advertisement
You might want to check this book out, I personally haven''t had a chance to read it yet but according to reviews on Amazon it seems to be pretty good, Game Scripting Mastery by Premier Press.

Share this post


Link to post
Share on other sites
It would be nice to see the table of contents and some sample pages, as I''m afraid it will only cover the basics and not help me with the parts I''m having trouble with. Anybody know if the book would help me?

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Also, check out "Virtual Machine Design and Implementation in C/C++" by Bill Blunden. It covers everything you''re looking for at the virtual machine level, but there''s no HLL included, just an assembler.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Look up ''Compilers: Principles, Techniques, and Tools'' by Aho, Sethi, and Ullman.

I am working on something that is similar. Feel free to check out my source code at

http://sourceforge.net/projects/saquen/

-D

Share this post


Link to post
Share on other sites
From my bookmarks: link.

While this isn''t about scripting engines, it will give you a complete understanding of how to implement things (like function calls) in a stack based environment.

Share this post


Link to post
Share on other sites
Extarius:
Gamescripting Mastery would definetly answer most of your questions. It details how to implement data types, stacks, method calls, and arrays. It doen't cover custom types though. On the whole it is a very good book, and I would definetly recommend it. The other books mentioned "Virtual Machine Design and Implementation in C/C++" and "Compilers: Principles, Techniques, and Tools" are very good as well. Yet for your needs I would definelty go with GameScripting Mastery.....


[edited by - PaleRaider on January 22, 2003 7:56:32 PM]

Share this post


Link to post
Share on other sites
I can really recommend the tutorial series at
http://www.peroxide.dk

Brilliant stuff and it implements int, bool and string types on a single stack (double type is easy to add, I''ve done it in under a half an hour). the tutorial on the VM isn''t finished yet though.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Is there a good reason you''re not using Lua or Python?

Life is shorter than you think...

Share this post


Link to post
Share on other sites
2 Main Reasons: I don't like either language, and I wouldn't learn as much using other people's code (not that I have anything against using libraries, but I like to understand whats going on under the hood first). Also, neither language would fit my needs as well as a language designed specifically to meet them.

I'm only 18, I've got the time =-P Would probably be a lot easier after I get my CS degree, but I don't want to wait till then (and really it might not help, so far I've taken 3 programming classes and none of them exposed me to anything new, taking data structures now and it doesn't seem like it will either since I've already messed around with linked lists, binary trees, sorting, etc)

[edited by - Extrarius on January 23, 2003 6:54:26 PM]

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
I would use Lua or Python either. I used perl for my scripting engine but it''s binaries were so huge to distribute.

Share this post


Link to post
Share on other sites
What order should I buy the books in? It will probably be a while before I can get them all, but implementing a good scripting language is an important project(only to me, but if I get it right the first time I can use it forever =-). That "Compilers: Principles, Techniques, and Tools" is expensive! Must be a text book..?

It wouldn''t be any fun using lua or pythin because I despise both from what I have seen of them. I feel the same way about java (and ironically I''m going to try to implement a subset of C++ anways =-). Perl might be ok, but I''ve never done any OO in perl and I''ve heard things like inheritence aren''t very pretty in perl.

Making the language object oriented design is important, and I''d like to be able to go as far as implementing overloading operators, virtual functions, properties, etc. Consider it C++, modified and interpreted =-) Also, I''d prefer to write it without using YACC, Lex, or simmilar programs. While they make it faster, they also generate ugly code that isn''t easy to understand or put in classes. I''m doing this primarily to learn, and I think I''ll learn best if I code it all myself.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
''Compilers: Principles and Techniques'' is used as a textbook in many places. It is pretty much the gold standard for this kind of thing. When writing an interpreter, it is invaluable for the parser/compiling backend. However, you will have to look for other sources when trying to figure out how to actually write the virtual machine.

Therefore, the order in which to buy the books depends on which way you want to take things. If you want to focus on the virtual machine first, figure out a byte code and get one of the scripting books mentioned here. If you want to first write the backend that turns the text into that byte code, get ''Compilers''.

Also note: Go to the nearest university library and get a card. You''ll probably find at least a couple of these books there and can find out which ones help you the most before paying any money.

-D

Share this post


Link to post
Share on other sites
try boost's spirit
looks really nice!
link







"I turned into a driveway I thought was mine and crashed into a tree I don't have."- a real insurance claim

[edited by - Amnesty on January 24, 2003 11:44:41 PM]

Share this post


Link to post
Share on other sites
Looking over boost's spirit, it looks pretty neat and easy to use. And the code using it is easy to read! I'd still like to learn to make a compiler entirely on my own, but I'll probably get the book on virtual machines first since I can use spirit for the high-level langauge => VM `assembler` converter for now.

Edit: Hrmm... Maybe not so easy to use... How do I compile it for windows? Add all 10000+ files to a VC project? They didn't even supply a plain makefile, just lots of scripts =-(

[edited by - Extrarius on January 25, 2003 11:11:53 AM]

Share this post


Link to post
Share on other sites
I use a virtual machine. After learning about ASM code it was really easy to create a simple scripting langauge. I made the assembler in VB too

I don't use stacks, all the instructions use 'registers' and 'RAM' to save information. It calls functions in the game to change the game world and to modify data. YOu don't really need a 'string' type, you can use pointers and string functions like C++

So basically keep it simple

[edited by - Cybertron on January 25, 2003 12:28:21 PM]

Share this post


Link to post
Share on other sites
I'm getting an error on line 75 of composite.hpp

74: template <typename IteratorT>
75: struct embed_trait< rule<IteratorT> >
76: {
77: typedef rule<IteratorT> const& type;
78: };

composite.hpp(75): error C2065: 'IteratorT' : undeclared identifier
composite.hpp(75): error C2687: cannot define a nested UDT of a template class out of line

The error description in the VS.Net help doesn't seem to fit, so I'm not sure how to fix it.
Is the boost spirit code not standard c++ or is .net just not getting it? Either way, any idea how I could fix it?

[edited by - extrarius on January 25, 2003 10:14:06 PM]

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
This is a VC++.net issue. If you look up the topic ''standard compliance issues in Visual C++'' in the help, you will see a section mentioning that it doesn''t support partial template specialization. Spirit seems to rely on this.

Therefore, you have a few options.
a) You can use gcc (I think it is compliant in this way).
b) You can switch to bison or yacc which are other programs that are meant to achieve the same purpose.
c) You can hand-craft your parser.

Note that spirit, yacc, and bison do not actually create your compiler for you. They just simplify one aspect of creating it. The disadvantage of bison and yacc is that they are geared towards c rather than c++ and therefore have various annoying things attached to themselves. i.e. lack of namespaces, defines everywhere, no support for classes, etc.

Having said that, there are ways to morph their output to be less onerous to c++.

Hmm. Having looked through this topic again, I''ve just realized that nobody has yet addressed your issues specifically. Rather we''ve referred you to a bunch of books. Here are a couple of things to keep in mind:

First of all, using a stack machine is probably really good since it simplifies a whole bunch of operations. There are a few ways to get around the ''one type rule''. You could use the ''any'' type from the boost library. You could use polymorphism to make a class hierarchy and then make a stack of polymorphic pointers/smart pointers. You could have a stack of raw memory, and an ancillary stack which represents what type the things are on the main stack and hence how much memory to push and pop (this can break down if you start making classes to put on the stack).

I chose to make a hierarchy. Here is a link to the smart pointer wrapper around it:

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/saquen/saquen/shared/ScriptVar.h?rev=1.4&content-type=text/vnd.viewcvs-markup

Here is a link to the base class for this hierarchy:

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/saquen/saquen/shared/ScriptVarSecret.h?rev=1.4&content-type=text/vnd.viewcvs-markup

If there are carriage returns, delete them before viewing this.

Secondly: Worry about getting a basic type system up and running. Once that it done, then any composite types (arrays, structs, classes etc.) can be added to the system. So wait on this one.

Third) See the above.

When I get around to making arrays and complex types, I''ll just have to extend my hierarchy.

Fourth, The best way to handle function calls is to create a special class called an ''activation record''. The record contains everything about a current instance of a function, including arguments, local variables, and a handle to the next bit of code to execute. Then when you are executing a script, push and pop instances of these classes.

Here is the header for my of an activation record. It has a pointer to what I call the ''ModelMethod'' which is the place where the actual code and various defaults are stored. It has a list of local variables (I don''t have arguments because I just copy them into the first locations of the local variable list). I''ve got an instruction pointer which tells me what thing should execute next.

In the container class for this, there is a stack of these which gets pushed and popped as functions call and return values.

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/saquen/saquen/run/Method.h?rev=1.3&content-type=text/vnd.viewcvs-markup

Hope this helps.

-D

Share this post


Link to post
Share on other sites
Thanks for the code AP, but unfortunately it as I don''t know much at all about compilers and it doesn''t have an explanation of what it''s doing, it wasn''t very helpful =-(

After looking around on the net some more hoping to find a tutorial I missed or somethin glike that, I finally decided to buy some books. I went to the Premier Press''s website and noticed they are the publisher of the worst comptuer book I''ve ever read, which makes me wary about buying Game Scripting Mastery.
Anybody want to vouch for it?

Does explain how the compiler actually works and why or does it just say "use these tools to make the different parts and feed them these template files, then edit line X to read __" like too many tutorials on the net do?

I''d much prefer the way of doing it to learning how to use a few tools. I realise the tools don''t make the compiler, but they generate 99% of the code (either that or lots of tutorials are leaving out 99% of the code for no reason when they make reference to the various compiler compiler tools).

Since I can''t get boost spirit to work I''ll probably be going for the virual machine book first (possibly in addition to Game Scripting Mastery), because a low-level language is better than none, and presumably I could always write a backend to a c compiler that generates my custom virtual assembler if there is no other way.

500 x 1

Share this post


Link to post
Share on other sites

  • Advertisement