Sign in to follow this  

Scripting, for you?

This topic is 4651 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I'm assuming most people reading this post are using or have used a scripting language in the past. I've used a few, SpiderMonkey, GameMonkey and Lua are the most notable mentions; there's a few more that look appealing, Squirrel, AngelScript, Python... and the list of available scripting langauges is endless. I was thinking about the design of the most popular languages today, were they designed for us or were they designed for the author's tastes, whims and pleasures? The biggest problem I have with most of the langaues I've tried is the scripting API; you know - that bit of code we have to write to bind our game data/logic to the scripting environment. As a developer, I don't think I pay too much attention to how the script syntax itself is specified, I'm a semi-smart guy, I'll work it out. The real thing I spend my time on is the binding, using the script API to link the scripted world with the native world. I'm starting to think that when creating a new language/environment, script designers are concerned with designing the syntax of the langauge first, they'll concenrate on the virtual machine internals, tweaking opcodes and managing stacks. It's my feeling that the programmer's API is an afterthought, something tacked onto the end of a masterpiece of VM design. Take the excellent beginner's text Game Scripting Mastery, the author doesn't really focus on the API design until quite late in the book and even then it's cursory. Where's the dedicated chapter on usability? Where's the discussion about how to access the machine? The Peroxide Tutorials are another example; it takes 13 separate tutorials to get to one concerning the actual developer interface to the script. Sure, we know the finer points about scanning, lexxing and virtual assembly, but why doesn't anyone really seem to care about who will be using the script; more importantly how they'll be using it? In the script langauges I've looked at, it seems that API access tends to be wrapped up in third party code wrappers. I've even written one for the language I use the most. So after that semi-rant, it got me thinking about a few questions: how do you access your scripting environment? - Are you happy with how you access your scripting environment? If so, why? What aspect of it do you find the best? - Do you grit your teeth everytime you have to hit the scripter's API? - Do you wrap it up in third party code? If so, why? What makes you use the wrapper? - Have you written a scripting language in the past? If so, where did you spend most of your attention? - Would you sacrifice speed for usability of the API? Or vice versa? - Do you know what your 'ideal' scripting environment is? - Do you think that the designer had the scripter or the developer in mind when they created their language? Is it impossible to satisfy both parties? I'd like to hear your views.... *** puts down the mic and wanders off to the bar

Share this post


Link to post
Share on other sites
Quote:

- Are you happy with how you access your scripting environment? If so, why? What aspect of it do you find the best?


In a word, no. I've got most of the various quirks (Lua with tolua) worked out, but there are still a number of things that irritate me, most notably passing parameters into the script. For example, say you are trying to use an object. The object might have a script hook attached to an ON_USE event that points to a script to execute when the object is used. However, I need some way of specifying which object is being used, ie if I have to manipulate the object itself in some fashion. I find Lua's native stack manipulations ugly and cumbersome to fiddle around with, but no wrapper tool I have tried makes it any easier. I've semi-settled on an ugly hack that involves using a context class with all static methods and members, with pointers to various types of objects so that I can perform things such as ContextClass:GetCurrentObject() in order to query the currently-being-used object from within the script. It's ugly, but it works so I do it.

Quote:

- Do you grit your teeth everytime you have to hit the scripter's API?


Yes. I cringe, too. In a very beaten-dog fashion.

Quote:

- Do you wrap it up in third party code? If so, why? What makes you use the wrapper?


I use tolua, not out of any particular brand loyalty, but just because it seems to be the easiest for me to use. But it's a pain in the ass, it really is. However, I am not going to go into the endless, dull repetitiveness of exporting interfaces by hand to Lua, not with as much interface code as I typically have to export.

Quote:

- Have you written a scripting language in the past? If so, where did you spend most of your attention?


I started at one point, but it could not hold my interest. I always hated the classwork on grammars, parsing, etc. in college anyway. What I wanted was the finished product, not the particular kind of grueling hell that building the language is for me. [grin]

Quote:

- Would you sacrifice speed for usability of the API? Or vice versa?


Yep.

Quote:

- Do you know what your 'ideal' scripting environment is?


I honestly have no idea, save that it makes it as easy to call into the script as it is to call engine code from the script. In Lua, it's easy to call engine code once it's exported, but it's not so easy (as described earlier) to pass parameters into the script or even call script functions. I do a lot of executing files as a means of calling scripts, because I detest having to go through the clunky and awkward API.

Quote:

- Do you think that the designer had the scripter or the developer in mind when they created their language? Is it impossible to satisfy both parties?


Lua definitely had the scripter in mind and not the developer. I'd say it would be possible to satisfy both parties, since the tasks involved are not mutually exclusive.

So, when can we expect v0.1 of EvoScript or whatever you're going to call it? [grin] (I assume by this post and other stuff that's what you're working toward in the long run?)

Share this post


Link to post
Share on other sites
Doesn't AngelCode allow you to call C++ methods directly without writing wrapper classes? It really isn't that hard to do, it's just a matter of sitting down and adding support for various compilers. If course if you want type/parameter safety you'll have to add a tool that takes advantage of SWIG and/or gcc-xml that will parse C++ code for you. Anyway, it isn't that hard, it's just a matter of doing it.

Share this post


Link to post
Share on other sites
Quote:
Original post by VertexNormal
In a word, no. I've got most of the various quirks (Lua with tolua) worked out, but there are still a number of things that irritate me, most notably passing parameters into the script.


Quote:
Original post by VertexNormal
However, I am not going to go into the endless, dull repetitiveness of exporting interfaces by hand to Lua, not with as much interface code as I typically have to export.


Quote:
Original post by VertexNormal
I honestly have no idea, save that it makes it as easy to call into the script as it is to call engine code from the script. In Lua, it's easy to call engine code once it's exported, but it's not so easy (as described earlier) to pass parameters into the script or even call script functions. I do a lot of executing files as a means of calling scripts, because I detest having to go through the clunky and awkward API.



I'm going to be mean here and clump your comments together and answer them at once. I share your annoyances, I don't think it stops and starts with Lua, it's certainly been the case in scripting languages I've used. I don't particularly enjoy having to make static methods and use C-like callbacks; I'm coding in C++ so why can't the scripting language be designed for me?

So from your comments and my personal experience, I'm definitely feeling that 'binding' the two is a pain (sidestepped the obvious pun there). Would you trade your scripting 'langauge' for a scripting 'environment'? A framework that requires you to abstract and build your non-core game entities and data, letting you use them in a standardised, uniform manner in your engine and also access them via 'script'?


Quote:
Original post by VertexNormal
So, when can we expect v0.1 of EvoScript or whatever you're going to call it? [grin] (I assume by this post and other stuff that's what you're working toward in the long run?)


Oh, I'm mainly just musing for now. Bringing together ideas, trying to examine the current situation and work out why scripting can sometimes be a real pain in the ass. Possibly come up with ideas to help ease it along.

Share this post


Link to post
Share on other sites
Quote:
Original post by evolutional
I'm coding in C++ so why can't the scripting language be designed for me?

Part of the problem is C/C++ is hidiously hard to get compile/run time symantics of the objects & functions. Wouldnt be so much easier todo bindings if you could do something like .NET reflection on everything in C++?

This would of course make it so much easier to invoke C++ stuff from the script. But you want it the other way around, you want the type information of inside the script and outside to match, so it is a simple matter to call a method on an interface(aka abstract class). This again is trivial todo with .NET reflection's codegen.

Share this post


Link to post
Share on other sites
Quote:
Original post by evolutional
As a developer, I don't think I pay too much attention to how the script syntax itself is specified, I'm a semi-smart guy, I'll work it out.


a) If the script syntax wasn't relevant, there probably wouldn't be more than 2 or 3 scripting languages out there. You'd have one optimised for speed, and one for expressiveness. So obviously people do find that it matters.

b) Many scripting languages are either aimed at non-programmers (ie. not-so-smart guys ;) ) or are aimed at lowering the amount of programming expertise needed to be useful, meaning straightforward syntax is important.

So I'd say syntax is paramount, rather than irrelevant.

Quote:
I'm starting to think that when creating a new language/environment, script designers are concerned with designing the syntax of the langauge first, they'll concenrate on the virtual machine internals, tweaking opcodes and managing stacks. It's my feeling that the programmer's API is an afterthought, something tacked onto the end of a masterpiece of VM design.


Well, I've said why I think the syntax is important. I can also see why the API/binding is considered the least important. You only perform a binding once, however the script syntax and VM performance are with you forever. It's like comparing an O(1) operation to an O(N) operation.

And your questions:
Are you happy with how you access your scripting environment? If so, why? What aspect of it do you find the best?

I hand-write bindings to Lua. I tend to consider automatic binders to be the wrong tool most of the time, giving you an interface that is too low level.

Do you grit your teeth everytime you have to hit the scripter's API?

A little. :) I don't mind the Lua API since it all makes sense. I just wish it was better documented, especially given the developers' tendency to make a lot of things deprecated with each new release, and often fail to document their replacements.

Do you wrap it up in third party code? If so, why? What makes you use the wrapper?

Not as of yet; I prefer to think about how the scripting language offers me some usability benefits over the host language, and code specific functions to enable that. (eg. instead of explicit loops through my C linked list where I perform functionality on each node, extract the whole list into a Lua table and provide a function that operates on that table)

Have you written a scripting language in the past? If so, where did you spend most of your attention?

I have a primitive system for my MUD. I think most of my time was spent writing the code for the various keywords (eg. if, unless, else, wait). Having said that, it doesn't support subroutines or function calls, and variables are typically macros which are expanded immediately rather than evaluated as part of an expression.

Would you sacrifice speed for usability of the API? Or vice versa?

This really does depend on the situation.

Do you know what your 'ideal' scripting environment is?

Python, with better security. ;) Actually I couldn't really say, as Python is perhaps one of the less enjoyable languages to embed.

And your last question I believe I addressed already.

Share this post


Link to post
Share on other sites
Quote:
Original post by evolutional
...I'm coding in C++ so why can't the scripting language be designed for me?
Put simply, C++ lacks a standard ABI, which means there is no reliable way to obtain C++ interfaces from compiled objects or executable binaries.

Consider that scripting .NET languages is trivial and seamless, so much so that there's an article on doing so in MSDN - and we all know that MSDN doesn't provide tutorials on anything that isn't generally accessible. The truth is that integrating "scripting" and statically compiled languages without reflection facilities is hard!

By the way, my "ideal" "scripting" language, at this point in time and relative to current programming technologies, would pretty much be a non-beta release of IronPython.

Share this post


Link to post
Share on other sites
I do agree with you guys that most scripting engines offer poor integration functionality. In this regard I want to shamelessly plug Andreas Jonsson's (hope I spelled that right) AngelScript because I think it's brilliant and it goes a long way at addressing the integration problem.

I've recently started using AngelScript and the reason I chose this scripting engine in favour of perhaps more mainstream engines such as Lua, is the rich functionality for integrating functions, classes with properties, methods, operator overloading etc. The beauty of AngelScript is that it does not require the embedder (silly term I admit) to write bridge functions to integrate host functionality into the script. For example functions and global variables can be integrated directly, and in most cases, existing classes can be directly registered as types within the language too. Also variables and function parameters can be of any type (class) as long the type is itself registered beforehand

AngelScript appeals to me because I can write object-based (if not object oriented) script code, that is, I do not need to do everything in terms of calling script functions with a bunch of native parameter types to get things done. The constraint (or virtue, depends on the POV) of AngelScript is that it is a strongly-typed language and it does not support variant types like Lua. As far as I am concerned, that's fine, as in my opinion it reduces run-time script errors... but that's just me of course!

Anyways, I encourage you to give it a good look, I think it's worth the time!

Share this post


Link to post
Share on other sites
I had the same problem with my design. Since I'm a far-from-experienced programmer I came up with a rather awkward way to bind C variables and functions with my language, by using the keyword "extern" the same way C does. In that way, one can declare functions and variables using 'extern' and just before code generation (after type-checking) they are linked using special functions. Of course, functions must be of a specific type:


// script API
typedef void (*externalfunc_t) (vmachine_t*);// each binded function takes a pointer to the virtual machine object
void LinkExternalFunc(const char* symbol, externalfunc_t func) {/*...*/ };// link an external function to the given name
void LinkExternalInt(const char* symbol, int* value) {/*...*/ };// link an external variable (int) to the given name

AST* parsefile(const char* filename);
int typecheck(ast* tree);
script_t* generatecode(ast* tree);
void Execute(script_t* script);



For example, consider the following script:


/*
A Sample Script
filename: "script.s"
*/
extern function LoadModel(string filename) : int;
extern function RenderModel(int model, int frame) : void;
extern int game_version;

function frame_draw(int frame) : void {
int model = LoadModel("model.3ds");
RenderModel(model, frame);
draw_text("version: %d", game_version);
}


The host C code would be:


// host program
void LoadModel(vmachine_t* vm) {/*...*/};
void RenderModel(vmachine_t* vm) {/*...*/};
int gameversion = 1;

void main (void) {
AST* tree = parsefile("script.s");
if(typecheck(tree) != OK)
return;
// link externals
LinkExternalFunc("LoadModel", &LoadModel);
LinkExternalFunc("RenderModel", &RenderModel);
LinkExternalInt("game_version", &gameversion);
// generate code
script_t* script = generatecode(tree);
Execute(script);
}





Actually, its somehow the same way you bind functions in lua, but because of the declarations inside the script, the programmer knows which functions will be binded when the script runs. Use of 'extern' in a declaration adds the symbol to the symbol table but leaves the address to the contents zero. If the
'LinkExternalX' functions had not been called, the 'generatecode' function would produce an error for the symbols that are unresolved (zero content address).
What do you think?

Share this post


Link to post
Share on other sites
Quote:
Original post by evolutional
I'm starting to think that when creating a new language/environment, script designers are concerned with designing the syntax of the langauge first, they'll concenrate on the virtual machine internals, tweaking opcodes and managing stacks. It's my feeling that the programmer's API is an afterthought, something tacked onto the end of a masterpiece of VM design. Take the excellent beginner's text Game Scripting Mastery, the author doesn't really focus on the API design until quite late in the book and even then it's cursory. Where's the dedicated chapter on usability? Where's the discussion about how to access the machine?

The Peroxide Tutorials are another example; it takes 13 separate tutorials to get to one concerning the actual developer interface to the script. Sure, we know the finer points about scanning, lexxing and virtual assembly, but why doesn't anyone really seem to care about who will be using the script; more importantly how they'll be using it? In the script langauges I've looked at, it seems that API access tends to be wrapped up in third party code wrappers. I've even written one for the language I use the most.


Hi, great thread! This is exactly what I was hoping to find.

I'm currently in the process of writing a script language and I must admit you are completely right, as far as I'm concerned. I thought it out all quite well, how to make the VM, the instructions, the compiler, etc. But I didn't think much about how to interface to C or C++, until I finally had a working VM and compiler and was confronted with the problem how to make a useful API for calling functions and methods from C / C++.

This might have to do with the complexity of such a project, maybe. You try to concentrate on the things that from development point are most important (getting a VM and a compiler put together that works and makes sense...)
Of course the whole API thing is important, too, but you don't need it in the early days of development, so you give it secondary priority.

I hope to get some good ideas about how a smart interfacing / embedding could be done from this forum and you guys, since I figure most of you sit on the other side of the table, i.e. are developers who write a native application and want to embed a scripting library. And maybe I can take advantage from the experiences you have made. ;)

Cheers,
Stefan


http://www.jewe.org

Share this post


Link to post
Share on other sites
I have been very annoyed by the lack of good binding between C++ and scripts. My solution to the problem, with windows Portable Executable files, is to "dll export" any functions which want to be usable by scripts. Then, have a tool open up the EXE and actually read through the export table and get the mangled symbol names. Using MSVCs dbghelp library you can unmangle the names and then parse the function signature to get the full name, the return type, calling convention, and paramters. I output a "header file" with this data and it will be used by the compiler to do static-checking of these functin invocations. You can also retrieve the function addresses from the EXE header.

I imagine any "compilation system" which supports libraries has similar functionality and the same thing could be done.

I haven't addressed invoking script functions from C++ yet though.

Share this post


Link to post
Share on other sites
I agree to the premise of this thread. My response to it was to limit the bridge between script and C code as much as possible. Basically, all the game is in script and all the user interface and other library-like code is on the C side. I took advantage of my C functions mostly using int and char* as parameters or return values so I was able to get away with writing my own very simple C->Lua binder. Also, calling Lua functions from C is rather painless for me - I have a function much like printf, which generates a lua code string, runs it and returns results - parameters and pointers for the return values are pushed on the stack as var_args.

So, I avoided dealing with the C - script interface, and I chose Lua because I liked the language.

Share this post


Link to post
Share on other sites

This topic is 4651 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this