Sign in to follow this  
tmack

[.net] Parsing a scripting language in C#..

Recommended Posts

I'm working on my own custom 2D Game engine, and I implemented Lua. Although LUA is very easy to work with in .NET, my problem is I can't really port it very well to other systems (Xbox, Windows Mobile), that I know of.. Not sure that's even possible with the .NET version? So I was thinking, how hard would it be to develop my own SIMPLE scripting language for my engine? It doesn't have to do ANYTHING special whats so ever. All it has to do is maybe perform loops and if statements. I don't need to declare variables, since it's for NPCs. Most COMPLEX scripts would be already implemented (like AI). Eventually, of course, it'd grow to be more complex but for now it needs to be simple. I'm thinking a simple script would be:
if (player.name == "hello") {
   player.setName = "hello!";
}

if (player.Flags["flagname"].value == "key1253") {
    someNpcName.OpenDoor = true;
    ShowMessage("You unlocked the door.");
}



That is literally all it would need, is the ability to set/get variables, if statements, loop statements (something like do until blah == blah), etc. Truthfully, I'm not 100% crazy over lua syntax. I like my example. My problem is, I'm not 100% sure HOW to approach this (haven't tried). It would also need to support nested if statements, But I'm sure it would work as long as I designed it right. I don't need code, just perhaps good articles that would get me going. I was thinking of using Regular Expressions, and parsing it like that which would work easily, but I'm not 100% sure it'll be fast enough (especially on slower single core systems). [Edited by - tmack on December 26, 2009 9:11:55 PM]

Share this post


Link to post
Share on other sites
I'm no language expert. I don't know the proper way to "write a language" as I've never read a book on the topic, though I ran into a similar situation while writing my engine. Here are the basic steps I came up with while writing my language.
(You'll have to google these for more info, its such a huge topic)

Not needing to deal with variables makes your job much easier... writing a stack is not that bad, but garbage collection... that gave me some headaches.

1) Tokenize the code

Basically you need to go through the code and convert it into tokens. Tokens consist of identifiers, operators, constants, ect... Once it is tokenizer however, it will be much easier to work with.

2) Build an abstract syntax tree (AST)

This is a tree of operators and operands, hard to explain, just look at this picture for an example of PHP in an AST: http://www.phpcompiler.org/doc/latest/_images/ifx5.jpg

3) Two options:
a) Iterate through the AST and execute as you go
b) Iterate through the AST and output byte code instead of executing it

At first I did (a) but found that in game it ran a bit slow, so I wrote a very simple set of bytecode instructions. Bytecode is very cool because after outputing it, you can go through and look for patterns, and do things like remove dead code, replace complex code, ect... (this is called optimization)

Share this post


Link to post
Share on other sites
There's a C# compiler in .NET; I forget where it is (they like to move it) and it might not be available for the mobile versions, and you need elevated privileges to use it, but that's an option.

There's also IronPython and IronRuby (amongst others) which are interpretable into MSIL.

And I'd also be a little surprised if there wasn't some project somewhere to provide LUA access on the systems you want.

... all that said, you'd probably want to use Expression Trees to do your code generation and if you use a pre-made parsing library or are familiar with a good parser generator that talks C# it shouldn't take that long at all.

Share this post


Link to post
Share on other sites
C# compiles to CIL bytecode... You can just use C# as your scripting language, and either embed Mono into your application, or write a CIL bytecode interpreter, or if your game is already C#, then you're already good to go.

This way you don't need to mess with parsing, compiling, optimizing, etc etc etc... If you aren't wanting to dedicate a lot of time to it, I would avoid trying to design a language.

Share this post


Link to post
Share on other sites
Creating a (scripting) language is always exciting when you start. If you don't have a deadline and you absolutely want to roll your own scripting language, do so; along the way you'll be almost forced to improve your coding habits using patterns in order to avoid repetitions and smelly code. It's not a bad thing.

However, I've been bitten many times by the allure of compilers etc and I strongly suggest you to think about the problem in a different way: you say you won't be able to port LuaInterface (I assume it's LuaInterface), and you're right. But how high is this in your priority list?

The bottom line is: abstract the "scripting" part of your game engine and use Lua for the moment. It'll be much easier to swap out implementations later on.


public interface IScripting
{
void Bind(Action action, string name);
void RunString(string chunk);
object this[string path] { get; set; }
}

IScripting script = new LuaInterface();
script.RunString("a = 10");
var value = script["myobject.myvalue"];



I know this does not answer your original question; but it can help you save some time now, and focus on less time-consuming features of your engine :)

Have fun all the same, coderchris already outlined a realistic workflow.

Share this post


Link to post
Share on other sites
OK, I have seen this posted many times. Yeah, LUA is great but it isn't your code. Here is my opinion on this....

Write your own code.

Common example
yadayada = param 1, param 2, param 3

Write a routine called SplitParams.
Your routine passes back
yadayada as the command
param 1 as parameter 1
param 2, param 3 etc...

Section off your data such as...
[sectionname]
Include a bool that says this is a section name, act accordingly.

Once you do that porting it to any language will be a breeze. I've done this for almost 10 years now and it has saved me from ever having to use any external library to read my scripts plus I can update it any time I want (try doing that to the LUA library).

Just my 2 cents worth

Share this post


Link to post
Share on other sites
Quote:
Original post by LancerSolurus
Yeah, LUA is great but it isn't your code.


That is... the entire point?

The instinct to not use something because you can't 'control' it or 'know what's going on' is something pretty much all programmers face, and need to overcome. Writing your own ini-like config just provides a lot of code to write and debug for no real gain. If you want some arbitrary formatted text to use with many languages, XML or even the ini format itself has parsers prebuilt in many languages.

Libraries are your friend. Writing code you don't need to write is just a waste of time and distracts you from actually getting the game done.

Share this post


Link to post
Share on other sites
I'm thinking LUA and any other scripting languages would be "too" much for me. Most of the AI would be hard coded, in C#, in the engine. All I need to do is to "call" the functions. There would be 100's of functions that you could call to do different things (move camera, move characters, etc) (movecam newx newy, movechar charRef newx newy, etc)

I'm worried, I go and implement LUA. I decide I want to port it to XBOX 360 - all I need to do is convert my SlimDX code to XNA, and I'm basically there. If my code has LUA, I'm pretty much screwed. I have to rewrite all my AI scripts...

And no, I have no deadline. And it would be really cool to have my own scripting language, and heck, it could turn into something 2-3 years from now.

Where can I read more about using C# as my scripting language? Couldn't I compile my scripts then? That would be the ideal way, but then again if it doesn't support it in XBOX/WM platforms then I'm beat. But wouldn't that open up a whole new world of exploits?

Thanks for all the great information =)

Share this post


Link to post
Share on other sites
So I'm thinking (watch out)....

If I was to use C# to compile the code, would this be true:

I could compile the game's scripts, and then simply execute them when the time calls for it? I could simply make a "scripting.dll" and only allow the scripter to access the functions that I implement (using set/gets, etc). Scripting.dll would simply cross reference into the actual engine.

To prevent exploits, could I not allow access to any namespaces, except my scripting.dll? If I want them to be able to access any kind of features (arrays, etc), I could just use set/gets to do so. Wouldn't this pretty much solve the issue of security? I'm not really sure how it works or how expandable this feature Microsoft provides, as I have not researched it yet.

Would that work? I'm just worry about security..

Share this post


Link to post
Share on other sites
So, a friend gave me a idea.. Here is what I wrote down on paper:

User creates "Level1a", and then adds two scripts named "SpriteTutCharAi" and "GetWeapon". User then creates "level1b" and adds script "ShowLevel".
When the levels are compiled, "Scripts.dll" is created by the level editor. The level creates the following classes and functions:
[source lang=cpp]
class level1a : ScriptingLibrary {
public void SpriteTutCharAi() {
ScriptTheUserWrote();
}
public void GetWeapon() {
DoActions();
}
}
class level1b : ScriptingLibrary {
public void ShowLevel() {
// Camera pans around the level to show it off
}
}



Basically, each time a "level" is created, a "class" is created for that level (class is the same name as the level), which inheirts the scripting library. The scripting library only allows access to certain features.

When the level is compiled, the script will be converted into a reference to the script inside of the DLL. The script will be executed once the conditions are valid (player touches the npc/sprite, enters level, onlevelload, etc.. this option will be given to the player when the script is created initially). The engine will then execute the code inside of the Scripts.dll library.

Would this idea work?

Share this post


Link to post
Share on other sites
Okay... going to read through each bit and reply where I can.

Quote:
Original post by tmack
I'm thinking LUA and any other scripting languages would be "too" much for me.


Trust me, working with scripting languages are nowhere near as annoying as stepping through parsing or compiler code that doesn't work.

Quote:

I'm worried, I go and implement LUA. I decide I want to port it to XBOX 360 -


Make something good first. And again, I'm reasonably certain that there's at least something floating around that will get LUA working on other platforms.

Quote:

Where can I read more about using C# as my scripting language?


There was an article about it here maybe 2-3 years back. Not sure if it was a proper article or a forum post with a link to another one. Finding the compiler in msdn and looking through it would be a start.

Quote:

Couldn't I compile my scripts then?


Sure, but that sort of defeats the purpose of scripts instead of code in the project...

Quote:

But wouldn't that open up a whole new world of exploits?


Of course; hence the permissions issue.

Quote:

This may be stupid or not even work, but what about using javascript and adding the functions I need? javascript is light weight, and fairly fast..


Sure. I'm not sure of what javascript bindings exist for the CLR, but there's probably something... It also depends if you're making a web-sort of game or not. Plus there's Silverlight which has better binding stuff and does the same sort of thing (afaik).

Quote:

I could compile the game's scripts, and then simply execute them when the time calls for it?


Sure. In general, they go one of three routes. One route compiles scripts into a CLR assembly. You then load the assembly and call stuff natively. Another compiles scripts into CLR delegates, which you can then pass around and call in code. The last simply interprets the script and 'does stuff' based on what it reads. The C# compiler in .NET does path one (IIRC).

Quote:

Wouldn't this pretty much solve the issue of security?

Would that work? I'm just worry about security..


I am not sure. It's been a while since I looked at it, and security wasn't any of my concern at the time. Personally, I think that requiring elevated privilege is a more immediate problem for something like a game.

Quote:

Would this idea work?


Sure, a lot of things can work. This seems a little awkward to me and seems to be a few steps ahead of where you should be looking. Find a technology, do some prototyping. See where it's good, see where it sucks. Then check your requirements and do some more prototyping. Does the technology fit well into your needs?

Share this post


Link to post
Share on other sites
Quote:
Original post by Telastyn
Okay... going to read through each bit and reply where I can.

Quote:
Original post by tmack
I'm thinking LUA and any other scripting languages would be "too" much for me.


Trust me, working with scripting languages are nowhere near as annoying as stepping through parsing or compiler code that doesn't work.

Quote:

I'm worried, I go and implement LUA. I decide I want to port it to XBOX 360 -


Make something good first. And again, I'm reasonably certain that there's at least something floating around that will get LUA working on other platforms.

Quote:

Where can I read more about using C# as my scripting language?


There was an article about it here maybe 2-3 years back. Not sure if it was a proper article or a forum post with a link to another one. Finding the compiler in msdn and looking through it would be a start.

Quote:

Couldn't I compile my scripts then?


Sure, but that sort of defeats the purpose of scripts instead of code in the project...

Quote:

But wouldn't that open up a whole new world of exploits?


Of course; hence the permissions issue.

Quote:

This may be stupid or not even work, but what about using javascript and adding the functions I need? javascript is light weight, and fairly fast..


Sure. I'm not sure of what javascript bindings exist for the CLR, but there's probably something... It also depends if you're making a web-sort of game or not. Plus there's Silverlight which has better binding stuff and does the same sort of thing (afaik).

Quote:

I could compile the game's scripts, and then simply execute them when the time calls for it?


Sure. In general, they go one of three routes. One route compiles scripts into a CLR assembly. You then load the assembly and call stuff natively. Another compiles scripts into CLR delegates, which you can then pass around and call in code. The last simply interprets the script and 'does stuff' based on what it reads. The C# compiler in .NET does path one (IIRC).

Quote:

Wouldn't this pretty much solve the issue of security?

Would that work? I'm just worry about security..


I am not sure. It's been a while since I looked at it, and security wasn't any of my concern at the time. Personally, I think that requiring elevated privilege is a more immediate problem for something like a game.

Quote:

Would this idea work?


Sure, a lot of things can work. This seems a little awkward to me and seems to be a few steps ahead of where you should be looking. Find a technology, do some prototyping. See where it's good, see where it sucks. Then check your requirements and do some more prototyping. Does the technology fit well into your needs?


Thank you! Right now I am looking prototyping my last post's idea. When you compile the level, the engine would simply just execute the function from within the script. This goes a bit above what I need to do, but I like it.

Wouldn't compiling the scripts at run time pretty much end the need for elevated privileges? You would only need elevated permissions for the level editor, which, upon first release would only be available to the developers.

Thank you very much for all your suggestions =). I really want to take my time on this, come up with the best ways to do things. My entire engine is going to be modular, pretty much. It'll support all kinds of different rendering routines. One cool feature with the level editor will be the ability to edit levels in "coop" mode with other level editors (in real time), along with a implementation of voice chat and text chat.

Right now I am working on the networking code, which is basically working VERY well. In fact, I just finished up rewriting some parts of it and now I'm off to test it again =)

I still would much rather write my own simple script, but I am thinking this is pretty complicated and I do really like the compile to a DLL idea.

*Another* idea (which would probably be slow, but I may concept it just for fun of it) is to compile all the level data into a DLL.

The engine would simply load the dll and all the scripting data would be there, compiled, ready to go. I guess it's just a crazy idea of mine that would never work =).

Share this post


Link to post
Share on other sites
Quote:
Original post by tmack
Wouldn't compiling the scripts at run time pretty much end the need for elevated privileges? You would only need elevated permissions for the level editor, which, upon first release would only be available to the developers.


No, unless I'm misunderstanding you. Just accessing the compiler requires elevated privilege (since you're essentially generating an assembly) last I saw. That was maybe 2 years ago now, so things might've changed.

Share this post


Link to post
Share on other sites
In situations where I need a simple script language I often use an old style of basic script like gwbasic as my basis.

The lines can be executed one line at a time, so they are perfect for fast interpreted langugae.

The interpreter can be created in less than 1000 lines of code.

Nesting can be almost completely avoided which makes the interpreter much easier to write and very simple to maintain.

http://en.wikipedia.org/wiki/BASIC

10 INPUT "What is your name: ", U$
20 PRINT "Hello "; U$
30 INPUT "How many stars do you want: ", N
40 S$ = ""
50 FOR I = 1 TO N
60 S$ = S$ + "*"
70 NEXT I
80 PRINT S$
90 INPUT "Do you want more stars? ", A$
100 IF LEN(A$) = 0 THEN GOTO 90
110 A$ = LEFT$(A$, 1)
120 IF A$ = "Y" OR A$ = "y" THEN GOTO 30
130 PRINT "Goodbye "; U$
140 END



A start to a basic compiler in C#, so you can see how quickly this type of thing can go together

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace basic
{
class Program
{
static void Main(string[] args)
{
string[] lines = System.IO.File.ReadAllLines(args[0]);

Dictionary<string, string> variables = new Dictionary<string,string>();
Dictionary<string, int> linePointer = new Dictionary<string, int>();
FillLinePointerTable(lines, linePointer);

for (int cp = 0; cp < lines.Length; cp++)
{
string line = lines[cp].Trim();

string[] tokens = line.Split(new char[] { ' ' });

switch (tokens[1].ToLower().Trim())
{
case "input":
System.Console.WriteLine(tokens[2]);
string result = System.Console.ReadLine();
string key = tokens[3];
variables[key] = result;
break;
case "print":
System.Console.WriteLine(tokens[2]);
break;
case "goto":
string c = tokens[2];
cp = linePointer[c];
break;
case "if":
//check condition
//true --> exeucte statement
//false --> move to else clause
break;
}

}

}

private static void FillLinePointerTable(string[] lines, Dictionary<string, int> linePointer)
{
for (int cp = 0; cp < lines.Length; cp++)
{
string line = lines[cp];
string[] tokens = line.Split(new char[] { ' ' });
linePointer.Add(tokens[0], cp);
}
}
}
}


Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this