Sign in to follow this  

Writing a scripting language

This topic is 4789 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

The language itself is an abstract concept that you decide on. A program which makes use of that language is made just like any other program, although software such as Lex and Yacc help to automate some of the well-understood stages of using a language (such as lexing and parsing).

Share this post


Link to post
Share on other sites
Keep one thing in mind: Implementing a non-trivial scripting language directly in C++ will be much harder than learning and using tools for lexing and parsing such as lex and yacc.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Don't listen to any of the guys .. check this link out ...

http://www.devincook.com/goldparser/

Share this post


Link to post
Share on other sites
Quote:
Original post by silvermace
i wrote my own compiler system in less than 3 weeks
its not complete, but its extremely usable.

check it out, its on these forumns (Tribute BASIC; in the
scriping and mod forumns)


realy?
i have had people tell me that it isnt posible

Share this post


Link to post
Share on other sites
im not saying its a miracle language or anything, but there
is very little it can't do - and those will definatly be addressed in the next release in 2 or so weeks

check it out.
http://www.gamedev.net/community/forums/topic.asp?topic_id=270108

PS. I'm not kidding about the 3 weeks, it will be 3 weeks on
thursday (tommorow here in NZ)

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Quote:
Original post by silvermace
im not saying its a miracle language or anything, but there
is very little it can't do - and those will definatly be addressed in the next release in 2 or so weeks

check it out.
http://www.gamedev.net/community/forums/topic.asp?topic_id=270108

PS. I'm not kidding about the 3 weeks, it will be 3 weeks on
thursday (tommorow here in NZ)


AND 1-2 months. for the VM......

depends how much of the functionality is in the VM versus complexity in the compilers code generator.

7-11 weeks in reality....

And then there is the testing/verifying/fixing all the feature
endcases. (There are more than a few, even in BASIC)



As to things usually needed for a script language --
how about interfacing to ASM/C/C++ libraries/calls to do graphics/engine interface stuff... which is critical for a real script language.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Quote:
Original post by Sneftel
Keep one thing in mind: Implementing a non-trivial scripting language directly in C++ will be much harder than learning and using tools for lexing and parsing such as lex and yacc.


Unfortunately that is only part of writing a compiler.
The language interactions and executable creation (VM bytecode or
native code) is still a major chunk of work those dont do.

(and if its bytecode, you still have to do the VM)

Share this post


Link to post
Share on other sites
Quote:
Original post by Sneftel
Keep one thing in mind: Implementing a non-trivial scripting language directly in C++ will be much harder than learning and using tools for lexing and parsing such as lex and yacc.


To be honest I found it easier to write a simple recursive descent parser than to work out how lex and yacc work, and that was with a background in Backus-Naur form and various grammars. Those tools are awkward and poorly-documented in my opinion. But maybe others find it to be the other way around.

Share this post


Link to post
Share on other sites
Quote:
Original post by Kylotan
Quote:
Original post by Sneftel
Keep one thing in mind: Implementing a non-trivial scripting language directly in C++ will be much harder than learning and using tools for lexing and parsing such as lex and yacc.


To be honest I found it easier to write a simple recursive descent parser than to work out how lex and yacc work, and that was with a background in Backus-Naur form and various grammars. Those tools are awkward and poorly-documented in my opinion. But maybe others find it to be the other way around.
I haven't finished my scripting language yet(and I'm not even sure I will), but so far I've found it extremely easy to manually create a recursive descent parser as well. I implemented a tokenizer without any assistance, and after reading a few chapters in the dragon book I already know how to completely implement the parser (its just actually doing it that is the problem - I have a tendency to lose interest once I know how to finish something).

Looking at Lex and Yacc, I find they do hardly and of the work for you really{like I said tokenizing is simple, and parsing isn't at all difficult once you have the basic idea down), and the code they make is useless except as a stand-alone program because its 'coding style' is horrendous(and I've heard people say there are some that generate nice code, but none of them look nice to me).

For making your own scripting language, I suggest reading any internet tutorials you can find (plenty on google) but also getting the "dragon book". You'll have to look in some of my past threads to get the actual title of the book because I'm being lazy right now.

Share this post


Link to post
Share on other sites
Game Scripting Mastery helped me immensely in understanding the basics of parsing a script language. It's not too hard to create a basic VM and assembly-like language, however the parsing of a higher level language seemed much harder. Alex Varanese does a decent job at providing a beginner's guide to parsing, IMHO.

Share this post


Link to post
Share on other sites
Quote:

realy?
i have had people tell me that it isnt posible


I wrote mine in some weeks too and even if it's not top-notch it's already quite advanced.

It has C syntax/scope, function, fully iterative, namespace, interfaces with C native functions, multithread native, output quite efficient bytecode (with optional multi-pass optimizer (5x AngelScript 1.8.0c speed, 3x GameMonkey speed computing the classic Mandelbrot set)).

It's fully C++ from scratch and the parser is *very* solid and has very good and precise warning/error reporting (I'm setting my standards to VS.Net being the top). The VM is just a stupid switch() like a Z80 emulator would have so it took some hours to write...

And I just hate (don't ask me, I really got allergic since KDE makefiles) everything GNU, they must have setup a "bloat teaching" school for the GNU developers. I'm sure those parser generators are very good (provided you have some weeks ahead of you to learn how to use them, have a look at GameMonkey's compiler I just can't believe how many errors slip in silently) but if you want to learn writing something then just write it. My opinion... I see too much hobby projects turning in LEGO games lately. That's no fun really. Sure, the goal is to DO something. Well, sometimes it isn't.

That said the things that gave me the most troubles:
-Parser: Cool/light scope handling with multiple implicit scopes was tricky.
-Linker: Some tricky checks for custom user types from different script objects.

So I guess it's quite possible but I'm not a code newbie either \:. Sorry to sounds arrogant, I just mean that in my 17 years of coding I must have made many indirect tries at it.

Share this post


Link to post
Share on other sites
check out the following:

Virtual Machine Design and Implementation in C++ by Blunden (Wordware)
Compilers: Primciples, Techniques and Tools by Aho et al (AdWes)
and Game Scripting Mastery by Varanese (Prima)

also, if you want to write an OO language (like UnrealScript) then an understanding of an Object Model Layout will help:

Inside the C++ Object Model by Stanley Lippman

[Edited by - algorhythmic on September 17, 2004 4:01:10 AM]

Share this post


Link to post
Share on other sites
you might not want to use any vm. you can directly compile on the fly your scripts to x86 bytecode (or whatever your target processor(s) is).
you'll need to parse and compile your scripts to x86 asm, and use or make a runtime JIT assembler, like softwire for example. (http://softwire.sourceforge.net/), although softwire is quite slow if you assemble big sources. most of the time will probably be spent in the compilation anyway, not in the assembly, so you can probably use the lib directly.

Share this post


Link to post
Share on other sites
if you want to do something nifty and non-standard like the UnrealScript system you will need to write your own VM though. I'm playing around with such a system using a quick and dirty LUA+luabind model to share object instance between script and engine. Once I've got the model fully defined I'll write my own VM+compiler system...

Share this post


Link to post
Share on other sites
Quote:
Original post by CodeTitan
Or can you just write it in C/C++?


Sure can, take a look at:
http://spirit.sourceforge.net/
or
http://hapy.sourceforge.net/

Define your grammer, and parse it, all in C++.

Share this post


Link to post
Share on other sites
And of course don't forget Boost.
I used one of the boost tokenize classes in the early stages of my script language.
Then I switched to a Boost::Spirit based compiler, which works very well.
IMHO Boost::Spirit is an absolutely incredible parsing framework (and check out Boost::Pheonix, Spirit's companion library), especially since it is entirely contained in header files, no object files at all!

Share this post


Link to post
Share on other sites
You could check out this tutorial series that I wrote:

http://www.peroxide.dk/tuts_scr.shtml

It should provide you with a good starting point for a C-style scripting language. First understand and master everything I write about in my tutorials, then expand the language with whatever features you need.

But as other has said - creating a full language and VM is quite a lot of work (but still very possible), so consider yourself warned :)

- Kasper

Share this post


Link to post
Share on other sites
Quote:
Original post by swiftcoder
IMHO Boost::Spirit is an absolutely incredible parsing framework

I thought Boost::Spirit is a very interesting proof of concept (the concept being advanced hackery of C++ language) but is absolutely unusable in real life. Compilation times being one of the major problems (although it should improve with the newer compilers). I also thought the architecture was an ugly piece of crap. I didn't understand much about templates back then, so I might want to go back to Spirit and reevaluate it again. In any case, to each his own [smile]

Share this post


Link to post
Share on other sites
I've written script interpreters and compilers for my own virtual machine. Even though there are tons out there, it is still a lot of fun. In fact, they are one of my most favorite things to write. I highly recommend the Let's Build a Compiler series that has already been mentioned.

Share this post


Link to post
Share on other sites
I think that one of the best things to do is to write a bnf form of the language you want to write. Even if you aren't going to use yacc or bison on this bnf it lets you get the language straight before doing anything.

As for learning lex and yacc (I use flex and bison), they aren't the easiest things to learn in the world but defiantly worth it. I first started writing my own parser / lexer and I though I did fairly well, but there is always a lot of duplicate code - even just if statements for error checking. I soon changed to lex / yacc because it was much better.

What I like about using lex and yacc is that you can be confident that it will only accept your language, I won't go into the details but it is mathematically proven. As well as the fact that you can't get a language acceptor any simpler than the bnf that describes it - and this is essentially what yacc uses to generate the parsing code.

Wikipedia Backus Naur Form

Share this post


Link to post
Share on other sites
The answer: don't. There are enough already. Use Lua, or Java Script, or a similar language. Only write a language if you're positive you need one - and if you are, you're a language expert, and wouldn't be asking this question.

Share this post


Link to post
Share on other sites
A bit blunt... It is true that there are plent of languages out there, and that writing a language is hard work but there can be an element of fun in it. I know that is one of the reasons I'm doing it.

Also the question wasn't about whether to write a scripting language or not, it was about how to do it. But you have a valid point none the less.

Share this post


Link to post
Share on other sites

This topic is 4789 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this