Sign in to follow this  
Ollhax

Unity Using whitelisting to run untrusted C# code safely

Recommended Posts

Ollhax    428

Hi there! I've been working on a proof-of-concept for a game-maker idea I've had for a while. It boils down to running user-written, untrusted C# code in a safe way. I've gone down the path of AppDomains and sandboxes, using Roslyn to build code on the fly and running the code in a separate process. I have a working implementation up and running, but I've hit some snags.

My biggest issue is that it seems like Microsoft have given up on sandboxing code. See https://msdn.microsoft.com/en-us/library/bb763046%28v=vs.110%29.aspx. They added the "Caution" box a few months back, including this gem: "We advise against loading and executing code of unknown origins without putting alternative security measures in place". To me, it feels like they've deprecated the whole thing.

There is also the issue that AppDomain sandboxing isn't very well supported across platforms. There's no support in Mono. I had hopes for a fix from the CoreCLR, but then I found this: https://github.com/dotnet/coreclr/issues/642 - so no luck there.

So! I've started exploring whitelisting as a security measure instead. I haven't  figured out how big a part of the .NET library I need to include yet, but it feels like I mainly need collections and some reflection stuff (probably limited to messing with public fields). I think I can do all this by examining the code with Roslyn and not allowing namespaces/classes that aren't explicitly listed.
 

I'm comparing my approach with Unity, which does more or less the same thing, e.g exposing only a safe subset of the framework. In their case it's an actual stripped down version of Mono (if I've understood it right), but seems to me the results would be pretty much the same if I get it right.

 

TLDR:

If you have experience with these kind of problems, would you say that is a safe approach? Am I missing something big and obvious here?

Share this post


Link to post
Share on other sites
Nik02    4348

With reflection, you can circumvent such whitelists.

 

In general, it is not wise to trust user-written code at all. It is unrealistic to assume that one could take into account all possible attack vectors. 

 

In "game makers", the user usually produces just data (or very primitive logic as in LBP), and STILL some users are able to hack the systems via said data.

 

Case in point, I played Mario Maker two days ago and I stumbled upon a level named something like "this will crash the game" - which it indeed did. It is not far-fetched to think that such level could then execute arbitrary code by using the level data as the injection vector, even though presumably the MM level system was not by any means designed to run user code.

 

Case in point 2: by trivially editing some save data of certain Wii games, one used to be able to cause a buffer overflow which could then be used to run arbitrary code, including overwriting the system firmware with a custom one loaded from USB or SD. The trivial edit? Change a save file name to be just slightly longer than the buffer allocated for it. The vulnerable games, which I won't mention here, were not particularly obscure either.

Edited by Nik02

Share this post


Link to post
Share on other sites
BitMaster    8651
I have little experience with C# but I have read into sandboxing a bit in the past. From my knowledge sandboxing works better in some languages than in others (and not at all in a few).

Lua for example should work pretty well because the language constructs available can't be reasonably used to break out of the sandbox or sabotage it. You have to be careful about which 'standard library' functions you give the user though (metatable manipulation and raw* functions are an obvious red flag, but they have a wiki page talking about it). Edited by BitMaster

Share this post


Link to post
Share on other sites
BitMaster    8651
Let's say it like this: I'm pretty certain you cannot break the sandbox without giving a malicious user access to the Lua standard library. The malicious user can still make the Lua VM run out of memory though and the host program has to handle that correctly (mishandled error conditions which usually do not happen are after all a very popular exploit vector). The malicious user can also deadlock a thread unless extra precautions are taken.
Whichever API you inject into Lua (and you have to inject something, unless you just want Lua as a glorified .ini file) must also handle being called in malicious ways.

I don't think you can get much better than Lua for sandboxing purposes though. Out of the box it gives you nothing dangerous and you can control exactly which piece of user code sees what. A lot of other languages can do something like an 'import <feature>' or have access to global objects which you cannot prevent without jumping through a lot of extra hoops.

Share this post


Link to post
Share on other sites
Alberth    9525

I have some major philosophical problems with the title, I don't see how you can run untrusted code safely in any way.

Either you trust the code and then by definition it's safe enough to run it, or you don't trust the code which means it is unsafe to run by definition. (The latter doesn't need to stop you from running it, it is mostly a slight shift towards expecting malicious behavior by default.)

 

People have been trying to keep crackers out of systems by doing anything you can imagine and a lot more you didn't yet think of, and so far, in the past 25-30 years, they failed. And that is about systems you can only access remotely, by network cable. You let code run on a CPU inside the machine itself! I believe you're just fooling yourself if you think it can be anywhere near safe if you don't trust the code.

 

If you want to make the C# code useful, you must give it some power, which means it is exploitable, even if it's just DoS-ing the local system. You can make it difficult to do harm, but in the end, a malicious person with enough motivation or knowledge cannot be stopped.

Share this post


Link to post
Share on other sites
BitMaster    8651

I have some major philosophical problems with the title, I don't see how you can run untrusted code safely in any way.
Either you trust the code and then by definition it's safe enough to run it, or you don't trust the code which means it is unsafe to run by definition. (The latter doesn't need to stop you from running it, it is mostly a slight shift towards expecting malicious behavior by default.)
 
People have been trying to keep crackers out of systems by doing anything you can imagine and a lot more you didn't yet think of, and so far, in the past 25-30 years, they failed. And that is about systems you can only access remotely, by network cable. You let code run on a CPU inside the machine itself! I believe you're just fooling yourself if you think it can be anywhere near safe if you don't trust the code.
 
If you want to make the C# code useful, you must give it some power, which means it is exploitable, even if it's just DoS-ing the local system. You can make it difficult to do harm, but in the end, a malicious person with enough motivation or knowledge cannot be stopped.


I strongly object to this line of thought. Let's look at StarCraft (the original) and Warcraft 3. A player could go online and browse for games to play with people. While both games came with premade maps, they also contained powerful map editors. When you entered a game with a map you did not have, you automatically downloaded it (as well as having a thriving ecosystem of sites where users could upload, comment and download maps). Apart from the pure map data the maps could also contain significant amount of scripts (after all, that's where the whole Defense of the Ancients concept came from: a Warcraft 3 map which ended up being popular).
That worked because the map scripts were properly sandboxed and to my knowledge there was never any exploit where the risk to a player was ever more than "it's not fun".

Granted, the OP's initial idea of using C# is probably not feasible (but I already wrote about that and talked about alternatives). Care must be taken to properly sandbox things but it has been done in the past and, especially if you only need a limited scope, it is doable.

Share this post


Link to post
Share on other sites
Ollhax    428

Thanks for the replies so far! I should have explained my situation a bit more. It's about the same as BitMaster's example of WC3 maps. I want to use C# for scripting-type of work. Even when limited, I expect it to be very useful. Some points for context:

  • Users will download mods as code and compile+run them locally. There's no downloading/running of arbitrary .exes or other files. I can examine the code thoroughly before running it.
  • I'll examine the actual semantic model of the code through Roslyn, not match raw code strings.
  • Disallowing the unsafe keyword should avoid problems with buffer overruns, etc. (Well, if I haven't missed something, which is why I'm posting this!)
  • Crashing isn't an issue. I can't help if a mod crashes the sandbox process, but it won't bring down the entire application at least. I imagine mods that crashes the game for you won't be that popular.
  • Allowing reflection isn't a requirement.

I'm interested to hear about specific ideas/examples for how you'd be able to attack this setup, given the constrains I mentioned above. I know it's a tricky thing to guarantee that something is secure, but at the same time I can't come up with a single concrete example in my setup where this would be an actual problem. If you'd like, consider it a challenge! smile.png

 

Side note: I use C# instead of Lua because I prefer that language, and I'm hoping to ride a bit on the XNA/Unity-wave. I can use Roslyn for real-time compiling, giving error messages, providing with intellisense-like tooling, etc. Also, it lets me use a common framework for my engine code and mod code. Basically, it saves me a *ton* of works, which makes this a feasible (more or less...) project for me.

Share this post


Link to post
Share on other sites
Brain    18906

Have you considered using a form of ecmascript or javascript as your scripting language? 

 

Javascript has been proven as a reasonable language to sandbox as it is used on websites the world over and it is extremely rare you get javascript exploits which allow execution of unsafe code or intrusion into other sandboxes. 

 

In the grand scheme of things, for the complexity and popularity of the language it is quite secure and there are literally licensed libraries for it...

Share this post


Link to post
Share on other sites
Nik02    4348
Roslyn cannot make the distinction between good and malicious intents. Something like encrypting all files in "my documents" is just business as usual for the system, but the majority of users certainly don't want a game to do that.

The overarching problem is that it is impossible to recognise harmful code in advance. And when all of your users get hacked by a malicious content pack that slipped past your inspection, it largely becomes your problem. Edited by Nik02

Share this post


Link to post
Share on other sites
Spazzarama    1643

Could you please elaborate on what you are trying to protect, your application, or your user's system?

 

I would probably read that caution as "don't rely solely on CAS".

 

You can provide a handler for the AppDomain resolve assembly name / type, and prevent particular types/assemblies from being accessed in your AppDomain. This might also help you prevent reflection from bypassing a whitelisting implementation (e.g. restrict access to the reflection types).

 

Btw, reflection can let you mess with non-public members as well, so don't ignore that.

 

Maybe all this combined with a "peer review/approved" approach to marking safe maps/plugins would be enough - then the user can be warned about the dangers of using a particular extension.

 

Add-ins might also be of interest: https://msdn.microsoft.com/en-us/library/bb384200.aspx and https://msdn.microsoft.com/en-us/library/bb355219.aspx

Edited by spazzarama

Share this post


Link to post
Share on other sites
Ollhax    428

Roslyn cannot make the distinction between good and malicious intents. Something like encrypting all files in "my documents" is just business as usual for the system, but the majority of users certainly don't want a game to do that.

The overarching problem is that it is impossible to recognise harmful code in advance. And when all of your users get hacked by a malicious content pack that slipped past your inspection, it largely becomes your problem.

 

Yes, that's right. I'm just using it to figure out what can potentially do that encryption (or whatever) - or rather, only enable stuff that I'm pretty sure cannot.

 

Could you please elaborate on what you are trying to protect, your application, or your user's system?

 

I would probably read that caution as "don't rely solely on CAS".

 

You can provide a handler for the AppDomain resolve assembly name / type, and prevent particular types/assemblies from being accessed in your AppDomain. This might also help you prevent reflection from bypassing a whitelisting implementation (e.g. restrict access to the reflection types).

 

Btw, reflection can let you mess with non-public members as well, so don't ignore that.

 

Maybe all this combined with a "peer review/approved" approach to marking safe maps/plugins would be enough - then the user can be warned about the dangers of using a particular extension.

 

Add-ins might also be of interest: https://msdn.microsoft.com/en-us/library/bb384200.aspx and https://msdn.microsoft.com/en-us/library/bb355219.aspx

 

I'm trying to protect the user's systems. They'll download mods in the form of code, compile it and run it locally. There won't be a central server that keeps the mods, at least not at first, so any security measures have to be done on the users' local machines.

 

As you say, CAS (or whatever the new security model is called) is still useful. I'll probably leave it in place as an added precaution for PC builds. However, I don't want to be limited to only PC releases, so I need an alternative as well.

 

You're completely right about runtime checking via assembly resolves, I have that check in place already. As far as I know, those are the only assemblies you'll be able to touch, in addition to the ones given to the compiler.

 

Reflection is tricky, agreed. Private member access may be hard to stop, so I'll have to think about that closely. I can probably make tools that let you do "safe" reflection, or just disallow it entirely.

 

Peer-reviewing is definitely a safeguard too. If a mod messes with your computer, you will probably report it, or at least not recommend it to others. But this is obviously only a last resort.

Edited by Ollhax

Share this post


Link to post
Share on other sites
WozNZ    2010

How much of the language have you white listed, what library code do you allow to be referenced and what do the APIs in your system that you have exposed allow manipulation of?

 

I believe unless you really clamp what is allowed there will be more holes than a teabag. The more you clamp the less useful your scripting will be and so the less people will want to use it.

 

I am with Alberth. The code is either un-trusted or trusted and if un-trusted there is no real way to run it in a safe way if a determined person wants to break out of "safe" playground. Some people will think that is a game in itself :)

If there is anything you have allowed that provides a whiff of system level access all bets are off. 

Share this post


Link to post
Share on other sites
WozNZ    2010

You will need to block expression trees. Give any access into this and you have huge troubles. Not sure how much of Linq that means you will need to block as that is the fundamental building block Linq is built on!!! 

 

Also you will need to block CodeDom, Reflection.Emit and also any other technique/library that allows run time code generation as all of these will get around your protection by allowing code generation after you have "checked" the code deemed it safe and then compiled it to run

Edited by WozNZ

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this