• Advertisement
Sign in to follow this  

Scripting Language Performance

This topic is 3865 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I've been tossing around the idea of doing an article that compares the performance of a hand full of scripting languages such as Lua, Game Monkey, Angel Script, Squirrel. I'm interested in setting up a head to head comparison as much as possible to attempt to give a good indication of their performance in different areas. I realize there are a lot of useless benchmarks that people use, so I'm looking for ideas for script snippets that would be more useful in getting a general idea how the languages stack up performance wise versus the others. Even though it could be considered useless to loop through tens of thousands of iterations of useless calculations and/or table manipulation, it would seem to me much more useful in a scripting language than it is for a native program, as it seems to me a good way to compare the general speed of the scripting languages instruction loop. For this reason I think it'd be useful to do some of these tests as well, but I'd also like to have some broader performance information than that provides, such as calls to and from native functions. Any ideas? Thanks.

Share this post


Link to post
Share on other sites
Advertisement
Quote:
Original post by DrEvil
I realize there are a lot of useless benchmarks that people use...

The comparative speed of scripting languages is a fairly useless benchmark. Why? Because speed is the lowest of the considerations as to why to use one language over another. More important considerations include ease of integration with other languages (as "scripting" languages are often used to extend "systems" languages); language expressiveness; ease of learning/use (as "scripting" languages are often placed in the hands of less-technical users to enable them quickly/easily extend a system or develop custom content); rich programming concepts exposed by the language (introspection, reflection, multiparadigm capability, closures, generators, decorators, etc); and so on.

All that said, the most meaningful benchmarking would be to run precisely the sorts of operations people switch to scripting languages for: introspection, reflection, dynamic object construction, closures, generators. Throw in member access/dispatch (because of the different ways the languages represent objects) and random access to and other operations on rich data types (dictionaries, lists, list slices, etc). That might actually be a benchmark worth arguing over. [smile]

Share this post


Link to post
Share on other sites


Finite State Machine mechanism

Game engine calls

Access to Game engine data (esp data conversion)

Context switching (sleep and resume) overhead

Attribute storage structures (list lookups -- including method lookup)

Action Queue building

Maximized Cache misses (via real engine execution to invalidate cache)

Share this post


Link to post
Share on other sites
Definitely test how fast you can send objects into the embedded engine and get objects out of it. In my (limited) experience, this is one area where scripts can incur a surprising amount of overhead.

Garbage collection. Is it spikey? Does it cause hiccups?

Share this post


Link to post
Share on other sites
Quote:
Original post by Oluseyi
The comparative speed of scripting languages is a fairly useless benchmark. Why? Because speed is the lowest of the considerations as to why to use one language over another.


Execution speed is of great importance to us developing for consoles, especially on Xbox 360 and PS3 where seemingly simple operations like casting a float to an integer can kill your performance.

Share this post


Link to post
Share on other sites
Quote:
Original post by Sly
Quote:
Original post by Oluseyi
The comparative speed of scripting languages is a fairly useless benchmark. Why? Because speed is the lowest of the considerations as to why to use one language over another.


Execution speed is of great importance to us developing for consoles, especially on Xbox 360 and PS3 where seemingly simple operations like casting a float to an integer can kill your performance.



I don't agree at all. One should use scripts only for non CPU intensive tasks, or go back to C/C++.
And I shipped PS1, Dreamcast, PS2, X360, and (upcoming this summer ) PS3 game with this principle.

Actually, to be completly honest, it is not that simple. Because one tends to put more and more code in the script part, as it is really faster to develop on, so we reach the limit where the code should be pass on C/C++.

Surprinsingly, it was most true on old consoles (PS1/dreamCast/PS2) that now.

The reason is that with old console, you defined very strict rules for what should be scripted and what should be in the low level engine. So in the end, the script job is only to pilot ( ie call functions and set /test flags ) the low level engine.
With nowadays consoles, you want to put much more content, for the same developpment time, so we tend to do much more things on the script part, including some that are quite lowlevel. And, in real life, it is not always easy to migrate some code from script side to engine side, because the code is not isolated enough.

In my experience, when you've reach this point, it is more a design issue than a language performance issue, and having a language with better performances will not prevent this.

On the other side, if there are a large amount of script code in your game, the quality of the language is VERY VERY important (cleaner -> less bugs -> more content or tuning ).

Hope it helps,

Emmanuel

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement