Jump to content

  • Log In with Google      Sign In   
  • Create Account


Different script solutions, performance and security


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
4 replies to this topic

#1 Ollhax   Members   -  Reputation: 156

Like
0Likes
Like

Posted 22 January 2012 - 03:00 PM

Hi there! As the title of the thread suggests, I'm currently trying out different ways of getting scripts to run in my game. The idea is to let users build their own game scenarios and such, similar to how Natural Selection 2 uses Lua for all its gameplay. I have some requirements:
  • The game is built with C# and XNA and the script should work with this setup.
  • The script should not be able to screw up the player's computer. I.e. no file access and such.
  • The script should run fast enough to not hinder my 60 fps target.
  • Hot-reloading the script would be nice. Not a 100% requirement, but it would help.
  • Good support for debuggers.
So, the main contenders so far are compiling C# on the fly and running Lua scripts. I'm not completely satisfied with either one.

Running C# scripts (i.e. just have .cs-files in a directory that are compiled on the fly when starting the game) sounded like the perfect solution at first. I like working in C#, and some quick testing revealed that the visual studio debugger can easilly attach to the script, the performance is great (equal to the rest of the code) and everything is peachy. Until the moment I realized that requirement #2 is violated, you can do anything with the scripts.

The solution I researched is to make a separate AppDomain, set minimal rights and run the script there. The problem is, the cross-domain performance is pretty awful. In my profiling tests, I get about 3500 calls to the script before 16 ms has passed.

So, I tried Lua. Using LuaInterface, I made the same kind of setup as before. My initial runs were positive, getting about 8x the performance of the C#/AppDomain scripts. Then I realized that I had to sandbox the lua calls too to make the tests fair, and after that the performance is much more evened out.

Stats when running 1000000 method calls, release build of course. Tests are:
  • simple: just a call to a function that returns a string.
  • noDomain: calling an identical function from a C# script. Most of the overhead is probably the reflection used to invoke the function.
  • domain: doing the same thing as above, but this time the script is run in a separate appDomain. I.e. real test for C# script.
  • lua: similar functionality for Lua instead.
Results in seconds:
  • simple: 0,0025514
  • noDomain: 0,6140648 (simple * 240,6776)
  • domain: 4,819073 (simple * 1888,795)
  • lua: 3,304015 (simple * 1294,981)
(1 / 60) / (domain's time / 1000000) = 3458 calls during 16.6 ms. Compare with ~5k calls from lua.

Question: Do these results seem reasonable? On average, how many script calls do you guys make per frame?

Sponsor:

#2 piupiu   Members   -  Reputation: 100

Like
0Likes
Like

Posted 22 January 2012 - 03:40 PM

If performance is an issue, you should probably use LuaJIT instead of the vanilla Lua implementation (it's a LOT faster).

Then I realized that I had to sandbox the lua calls too to make the tests fair, and after that the performance is much more evened out.

What do you mean by "sandboxing the lua calls"? This should not come with *any* noticeable performance hit...
I assume you just create a Lua environment with only the basic functions (print, pairs, etc.) so the sandboxed code can't access anything and then make sure it doesn't run for too long (infinite loops etc.)? (also see: http://lua-users.org/wiki/SandBoxes)

#3 Ollhax   Members   -  Reputation: 156

Like
0Likes
Like

Posted 22 January 2012 - 04:38 PM

Well, I'm unsure right now if performance will be an issue or not. I took a look at LuaJIT before, not sure how well it integrates with C#. Has anyone here tried it?

About sandboxing lua calls, I used the method described in the link you gave. Here's the code:

function ReturnString(x)
  s = "gamma"
  return s
end

local env = {print=print}
function run()
	local untrusted_function = ReturnString
	setfenv(untrusted_function, env)
	err, value = pcall(untrusted_function)
	return value
end

I'm not too familiar with Lua yet so the code may be suboptimal. I just call the run() method, which (as you see) in turn runs the untrusted code. In the final version, the run method will of course not be included in the script file.

#4 Vfor Vikram   Members   -  Reputation: 110

Like
0Likes
Like

Posted 29 February 2012 - 12:44 AM

Won't it be easier to have a black-list of namespaces like System.IO - find the references and just inform user "uknown class".

U can also explore how Unity3D is doing..

#5 Ollhax   Members   -  Reputation: 156

Like
0Likes
Like

Posted 29 February 2012 - 02:51 AM

Won't it be easier to have a black-list of namespaces like System.IO - find the references and just inform user "uknown class".

U can also explore how Unity3D is doing..


Hrm, I don't trust that solution. It seems like a very fragile defense, easy to poke holes in - especially since the "hackers" would have full access to the black listing code through reflector. I'd assume Unity just trusts the scripts, or perhaps the Mono implementation allows for extra security settings that are easier to use. It seems to me a different thing having developers making scripts in Unity vs users making scripts in my game, where the former can be trusted more or less (you implicitly trust every developer you install a game from) and the latter can't be trusted at all.

Right now, I've gone with the sandboxed C# script. I realized that I only have to make a few calls to the script per frame if I keep most of the game code in the script. Even though the state data sent between the sandbox and primary domain will be large, further testing indicated that you can send several megabytes before there's any real performance impact there.




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS