Scripting Language Performance
Author
I've been tossing around the idea of doing an article that compares the performance of a hand full of scripting languages such as Lua, Game Monkey, Angel Script, Squirrel. I'm interested in setting up a head to head comparison as much as possible to attempt to give a good indication of their performance in different areas. I realize there are a lot of useless benchmarks that people use, so I'm looking for ideas for script snippets that would be more useful in getting a general idea how the languages stack up performance wise versus the others.
Even though it could be considered useless to loop through tens of thousands of iterations of useless calculations and/or table manipulation, it would seem to me much more useful in a scripting language than it is for a native program, as it seems to me a good way to compare the general speed of the scripting languages instruction loop. For this reason I think it'd be useful to do some of these tests as well, but I'd also like to have some broader performance information than that provides, such as calls to and from native functions.
Any ideas?
Thanks.
Quote:Original post by DrEvil
I realize there are a lot of useless benchmarks that people use...
The comparative speed of scripting languages is a fairly useless benchmark. Why? Because speed is the lowest of the considerations as to why to use one language over another. More important considerations include ease of integration with other languages (as "scripting" languages are often used to extend "systems" languages); language expressiveness; ease of learning/use (as "scripting" languages are often placed in the hands of less-technical users to enable them quickly/easily extend a system or develop custom content); rich programming concepts exposed by the language (introspection, reflection, multiparadigm capability, closures, generators, decorators, etc); and so on.
All that said, the most meaningful benchmarking would be to run precisely the sorts of operations people switch to scripting languages for: introspection, reflection, dynamic object construction, closures, generators. Throw in member access/dispatch (because of the different ways the languages represent objects) and random access to and other operations on rich data types (dictionaries, lists, list slices, etc). That might actually be a benchmark worth arguing over. [smile]
Finite State Machine mechanism
Game engine calls
Access to Game engine data (esp data conversion)
Context switching (sleep and resume) overhead
Attribute storage structures (list lookups -- including method lookup)
Action Queue building
Maximized Cache misses (via real engine execution to invalidate cache)
Definitely test how fast you can send objects into the embedded engine and get objects out of it. In my (limited) experience, this is one area where scripts can incur a surprising amount of overhead.
Garbage collection. Is it spikey? Does it cause hiccups?
Garbage collection. Is it spikey? Does it cause hiccups?
Quote:Original post by Oluseyi
The comparative speed of scripting languages is a fairly useless benchmark. Why? Because speed is the lowest of the considerations as to why to use one language over another.
Execution speed is of great importance to us developing for consoles, especially on Xbox 360 and PS3 where seemingly simple operations like casting a float to an integer can kill your performance.
Quote:Original post by SlyQuote:Original post by Oluseyi
The comparative speed of scripting languages is a fairly useless benchmark. Why? Because speed is the lowest of the considerations as to why to use one language over another.
Execution speed is of great importance to us developing for consoles, especially on Xbox 360 and PS3 where seemingly simple operations like casting a float to an integer can kill your performance.
I don't agree at all. One should use scripts only for non CPU intensive tasks, or go back to C/C++.
And I shipped PS1, Dreamcast, PS2, X360, and (upcoming this summer ) PS3 game with this principle.
Actually, to be completly honest, it is not that simple. Because one tends to put more and more code in the script part, as it is really faster to develop on, so we reach the limit where the code should be pass on C/C++.
Surprinsingly, it was most true on old consoles (PS1/dreamCast/PS2) that now.
The reason is that with old console, you defined very strict rules for what should be scripted and what should be in the low level engine. So in the end, the script job is only to pilot ( ie call functions and set /test flags ) the low level engine.
With nowadays consoles, you want to put much more content, for the same developpment time, so we tend to do much more things on the script part, including some that are quite lowlevel. And, in real life, it is not always easy to migrate some code from script side to engine side, because the code is not isolated enough.
In my experience, when you've reach this point, it is more a design issue than a language performance issue, and having a language with better performances will not prevent this.
On the other side, if there are a large amount of script code in your game, the quality of the language is VERY VERY important (cleaner -> less bugs -> more content or tuning ).
Hope it helps,
Emmanuel
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
