Sign in to follow this  
Ollhax

Unity Using whitelisting to run untrusted C# code safely

Recommended Posts

Hi there! I've been working on a proof-of-concept for a game-maker idea I've had for a while. It boils down to running user-written, untrusted C# code in a safe way. I've gone down the path of AppDomains and sandboxes, using Roslyn to build code on the fly and running the code in a separate process. I have a working implementation up and running, but I've hit some snags.

My biggest issue is that it seems like Microsoft have given up on sandboxing code. See https://msdn.microsoft.com/en-us/library/bb763046%28v=vs.110%29.aspx. They added the "Caution" box a few months back, including this gem: "We advise against loading and executing code of unknown origins without putting alternative security measures in place". To me, it feels like they've deprecated the whole thing.

There is also the issue that AppDomain sandboxing isn't very well supported across platforms. There's no support in Mono. I had hopes for a fix from the CoreCLR, but then I found this: https://github.com/dotnet/coreclr/issues/642 - so no luck there.

So! I've started exploring whitelisting as a security measure instead. I haven't  figured out how big a part of the .NET library I need to include yet, but it feels like I mainly need collections and some reflection stuff (probably limited to messing with public fields). I think I can do all this by examining the code with Roslyn and not allowing namespaces/classes that aren't explicitly listed.
 

I'm comparing my approach with Unity, which does more or less the same thing, e.g exposing only a safe subset of the framework. In their case it's an actual stripped down version of Mono (if I've understood it right), but seems to me the results would be pretty much the same if I get it right.

 

TLDR:

If you have experience with these kind of problems, would you say that is a safe approach? Am I missing something big and obvious here?

Share this post


Link to post
Share on other sites

With reflection, you can circumvent such whitelists.

 

In general, it is not wise to trust user-written code at all. It is unrealistic to assume that one could take into account all possible attack vectors. 

 

In "game makers", the user usually produces just data (or very primitive logic as in LBP), and STILL some users are able to hack the systems via said data.

 

Case in point, I played Mario Maker two days ago and I stumbled upon a level named something like "this will crash the game" - which it indeed did. It is not far-fetched to think that such level could then execute arbitrary code by using the level data as the injection vector, even though presumably the MM level system was not by any means designed to run user code.

 

Case in point 2: by trivially editing some save data of certain Wii games, one used to be able to cause a buffer overflow which could then be used to run arbitrary code, including overwriting the system firmware with a custom one loaded from USB or SD. The trivial edit? Change a save file name to be just slightly longer than the buffer allocated for it. The vulnerable games, which I won't mention here, were not particularly obscure either.

Edited by Nik02

Share this post


Link to post
Share on other sites
I have little experience with C# but I have read into sandboxing a bit in the past. From my knowledge sandboxing works better in some languages than in others (and not at all in a few).

Lua for example should work pretty well because the language constructs available can't be reasonably used to break out of the sandbox or sabotage it. You have to be careful about which 'standard library' functions you give the user though (metatable manipulation and raw* functions are an obvious red flag, but they have a wiki page talking about it). Edited by BitMaster

Share this post


Link to post
Share on other sites
Let's say it like this: I'm pretty certain you cannot break the sandbox without giving a malicious user access to the Lua standard library. The malicious user can still make the Lua VM run out of memory though and the host program has to handle that correctly (mishandled error conditions which usually do not happen are after all a very popular exploit vector). The malicious user can also deadlock a thread unless extra precautions are taken.
Whichever API you inject into Lua (and you have to inject something, unless you just want Lua as a glorified .ini file) must also handle being called in malicious ways.

I don't think you can get much better than Lua for sandboxing purposes though. Out of the box it gives you nothing dangerous and you can control exactly which piece of user code sees what. A lot of other languages can do something like an 'import <feature>' or have access to global objects which you cannot prevent without jumping through a lot of extra hoops.

Share this post


Link to post
Share on other sites

I have some major philosophical problems with the title, I don't see how you can run untrusted code safely in any way.

Either you trust the code and then by definition it's safe enough to run it, or you don't trust the code which means it is unsafe to run by definition. (The latter doesn't need to stop you from running it, it is mostly a slight shift towards expecting malicious behavior by default.)

 

People have been trying to keep crackers out of systems by doing anything you can imagine and a lot more you didn't yet think of, and so far, in the past 25-30 years, they failed. And that is about systems you can only access remotely, by network cable. You let code run on a CPU inside the machine itself! I believe you're just fooling yourself if you think it can be anywhere near safe if you don't trust the code.

 

If you want to make the C# code useful, you must give it some power, which means it is exploitable, even if it's just DoS-ing the local system. You can make it difficult to do harm, but in the end, a malicious person with enough motivation or knowledge cannot be stopped.

Share this post


Link to post
Share on other sites

I have some major philosophical problems with the title, I don't see how you can run untrusted code safely in any way.
Either you trust the code and then by definition it's safe enough to run it, or you don't trust the code which means it is unsafe to run by definition. (The latter doesn't need to stop you from running it, it is mostly a slight shift towards expecting malicious behavior by default.)
 
People have been trying to keep crackers out of systems by doing anything you can imagine and a lot more you didn't yet think of, and so far, in the past 25-30 years, they failed. And that is about systems you can only access remotely, by network cable. You let code run on a CPU inside the machine itself! I believe you're just fooling yourself if you think it can be anywhere near safe if you don't trust the code.
 
If you want to make the C# code useful, you must give it some power, which means it is exploitable, even if it's just DoS-ing the local system. You can make it difficult to do harm, but in the end, a malicious person with enough motivation or knowledge cannot be stopped.


I strongly object to this line of thought. Let's look at StarCraft (the original) and Warcraft 3. A player could go online and browse for games to play with people. While both games came with premade maps, they also contained powerful map editors. When you entered a game with a map you did not have, you automatically downloaded it (as well as having a thriving ecosystem of sites where users could upload, comment and download maps). Apart from the pure map data the maps could also contain significant amount of scripts (after all, that's where the whole Defense of the Ancients concept came from: a Warcraft 3 map which ended up being popular).
That worked because the map scripts were properly sandboxed and to my knowledge there was never any exploit where the risk to a player was ever more than "it's not fun".

Granted, the OP's initial idea of using C# is probably not feasible (but I already wrote about that and talked about alternatives). Care must be taken to properly sandbox things but it has been done in the past and, especially if you only need a limited scope, it is doable.

Share this post


Link to post
Share on other sites

Thanks for the replies so far! I should have explained my situation a bit more. It's about the same as BitMaster's example of WC3 maps. I want to use C# for scripting-type of work. Even when limited, I expect it to be very useful. Some points for context:

  • Users will download mods as code and compile+run them locally. There's no downloading/running of arbitrary .exes or other files. I can examine the code thoroughly before running it.
  • I'll examine the actual semantic model of the code through Roslyn, not match raw code strings.
  • Disallowing the unsafe keyword should avoid problems with buffer overruns, etc. (Well, if I haven't missed something, which is why I'm posting this!)
  • Crashing isn't an issue. I can't help if a mod crashes the sandbox process, but it won't bring down the entire application at least. I imagine mods that crashes the game for you won't be that popular.
  • Allowing reflection isn't a requirement.

I'm interested to hear about specific ideas/examples for how you'd be able to attack this setup, given the constrains I mentioned above. I know it's a tricky thing to guarantee that something is secure, but at the same time I can't come up with a single concrete example in my setup where this would be an actual problem. If you'd like, consider it a challenge! smile.png

 

Side note: I use C# instead of Lua because I prefer that language, and I'm hoping to ride a bit on the XNA/Unity-wave. I can use Roslyn for real-time compiling, giving error messages, providing with intellisense-like tooling, etc. Also, it lets me use a common framework for my engine code and mod code. Basically, it saves me a *ton* of works, which makes this a feasible (more or less...) project for me.

Share this post


Link to post
Share on other sites

Have you considered using a form of ecmascript or javascript as your scripting language? 

 

Javascript has been proven as a reasonable language to sandbox as it is used on websites the world over and it is extremely rare you get javascript exploits which allow execution of unsafe code or intrusion into other sandboxes. 

 

In the grand scheme of things, for the complexity and popularity of the language it is quite secure and there are literally licensed libraries for it...

Share this post


Link to post
Share on other sites
Roslyn cannot make the distinction between good and malicious intents. Something like encrypting all files in "my documents" is just business as usual for the system, but the majority of users certainly don't want a game to do that.

The overarching problem is that it is impossible to recognise harmful code in advance. And when all of your users get hacked by a malicious content pack that slipped past your inspection, it largely becomes your problem. Edited by Nik02

Share this post


Link to post
Share on other sites

Could you please elaborate on what you are trying to protect, your application, or your user's system?

 

I would probably read that caution as "don't rely solely on CAS".

 

You can provide a handler for the AppDomain resolve assembly name / type, and prevent particular types/assemblies from being accessed in your AppDomain. This might also help you prevent reflection from bypassing a whitelisting implementation (e.g. restrict access to the reflection types).

 

Btw, reflection can let you mess with non-public members as well, so don't ignore that.

 

Maybe all this combined with a "peer review/approved" approach to marking safe maps/plugins would be enough - then the user can be warned about the dangers of using a particular extension.

 

Add-ins might also be of interest: https://msdn.microsoft.com/en-us/library/bb384200.aspx and https://msdn.microsoft.com/en-us/library/bb355219.aspx

Edited by spazzarama

Share this post


Link to post
Share on other sites

Roslyn cannot make the distinction between good and malicious intents. Something like encrypting all files in "my documents" is just business as usual for the system, but the majority of users certainly don't want a game to do that.

The overarching problem is that it is impossible to recognise harmful code in advance. And when all of your users get hacked by a malicious content pack that slipped past your inspection, it largely becomes your problem.

 

Yes, that's right. I'm just using it to figure out what can potentially do that encryption (or whatever) - or rather, only enable stuff that I'm pretty sure cannot.

 

Could you please elaborate on what you are trying to protect, your application, or your user's system?

 

I would probably read that caution as "don't rely solely on CAS".

 

You can provide a handler for the AppDomain resolve assembly name / type, and prevent particular types/assemblies from being accessed in your AppDomain. This might also help you prevent reflection from bypassing a whitelisting implementation (e.g. restrict access to the reflection types).

 

Btw, reflection can let you mess with non-public members as well, so don't ignore that.

 

Maybe all this combined with a "peer review/approved" approach to marking safe maps/plugins would be enough - then the user can be warned about the dangers of using a particular extension.

 

Add-ins might also be of interest: https://msdn.microsoft.com/en-us/library/bb384200.aspx and https://msdn.microsoft.com/en-us/library/bb355219.aspx

 

I'm trying to protect the user's systems. They'll download mods in the form of code, compile it and run it locally. There won't be a central server that keeps the mods, at least not at first, so any security measures have to be done on the users' local machines.

 

As you say, CAS (or whatever the new security model is called) is still useful. I'll probably leave it in place as an added precaution for PC builds. However, I don't want to be limited to only PC releases, so I need an alternative as well.

 

You're completely right about runtime checking via assembly resolves, I have that check in place already. As far as I know, those are the only assemblies you'll be able to touch, in addition to the ones given to the compiler.

 

Reflection is tricky, agreed. Private member access may be hard to stop, so I'll have to think about that closely. I can probably make tools that let you do "safe" reflection, or just disallow it entirely.

 

Peer-reviewing is definitely a safeguard too. If a mod messes with your computer, you will probably report it, or at least not recommend it to others. But this is obviously only a last resort.

Edited by Ollhax

Share this post


Link to post
Share on other sites

How much of the language have you white listed, what library code do you allow to be referenced and what do the APIs in your system that you have exposed allow manipulation of?

 

I believe unless you really clamp what is allowed there will be more holes than a teabag. The more you clamp the less useful your scripting will be and so the less people will want to use it.

 

I am with Alberth. The code is either un-trusted or trusted and if un-trusted there is no real way to run it in a safe way if a determined person wants to break out of "safe" playground. Some people will think that is a game in itself :)

If there is anything you have allowed that provides a whiff of system level access all bets are off. 

Share this post


Link to post
Share on other sites

You will need to block expression trees. Give any access into this and you have huge troubles. Not sure how much of Linq that means you will need to block as that is the fundamental building block Linq is built on!!! 

 

Also you will need to block CodeDom, Reflection.Emit and also any other technique/library that allows run time code generation as all of these will get around your protection by allowing code generation after you have "checked" the code deemed it safe and then compiled it to run

Edited by WozNZ

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this  

  • Announcements

  • Forum Statistics

    • Total Topics
      628394
    • Total Posts
      2982431
  • Similar Content

    • By toadvine
      I'd like to start building a 2D game, very small, just pixel art. I know python/javascript/tiny bit of HTML and I've started learning C++. I'm a long way from starting officially, I've just been making mini-projects to practice & drawing up concept art/gameplay storyboards. I'm just wondering if I'm off with the C++ for the game's programming, or if I should know certain things before starting.
      The game would be very small, I just want to get started trying out some actual work before college.
      Some other info: I'm working by myself on this, and going into game design/programming next year for college. I'm a high school student with a few years of experience just working by myself making small projects, drawing up stuff, so I really don't know much and I'd appreciate any tips.
      Thanks a bunch!
    • By Daerst

      SWARMED is a Zombie-themed RPG / RTS currently in development using Unity 3D. We love Dwarf Fortress (though we have no illusions that SWARMED will reach the same level of complexity), roguelikes, old-school point & click RPGs and real-time strategy games. We aim to cross genre-borders here and there and give some twists to the old Martinis every gamer has been drinking since the 1980s, metaphorically.
      Single player, 3D graphics and adjustable top-down camera - old-school RPG / RTS feeling Take control of a core group of survivors after the outbreak Encounter Zombies that are a real threat, no machine-gun massacre. Don't get swarmed! Build a safe zone anywhere with a highly flexible build system: campsite, lighthouse, school, or fence a whole village Grant asylum to other survivors that you meet and make them a part of your community Achieve sustainability in your safe zone and go on supply runs with your survivors
      Development
      The core team of recently founded indie studio Three Eyed Games currently consists of one writer, two artists and two programmers, based in Germany. We are in our mid-20s with professional experience in developing interactive 3D applications with Unity.
      SWARMED will feature both a 'free-play mode' and a campaign with mid-sized maps that leads the player through a story while explaining the gameplay and introducing him / her to the survivors: a core group a few 'hero' characters the player starts with (each one a detailed character with backstory, hopes and dreams), and more 'heroes' (total not more than 20, probably less) that the player can meet on the journey. In addition, randomly generated NPCs (less detailed and not directly controllable, similar to the way Dwarf Fortress handles its dwarves) can join your safe zone – if you let them.
      We plan to release a few 'Origin' prototypes that showcase individual gameplay systems and meanwhile give a gentle introduction to the characters you will meet in the game. Origin I, showcasing the build system and many fundamental elements like character controls and interactions, is finished and will be released soon. Next up, we're working on the dialog system to be presented in Origin II. Get in touch and we will provide more details and a playable version.
      We want you!
      We seriously think you should join the fun! We are looking for:
      Level Designers / Environment Artists, preferably with experience in Unity and procedural asset creation. Design and build maps with interesting visuals and proper pacing. 3D Artists. Our shacks, items and the dead guys' faces could use some plastic surgery. Can you do that? Writers. We have a bunch of characters to detail and a story to write ahead of us. Game Designers. We have a rough game design sketched out that needs improvement and completion. We need a balanced combat system, trees for constructions, workshops and character skills etc. PR & Community Managers, preferably with web development experience. We want to build a community around the game, and we need you to plan and manage this (with the help of the rest of the team, of course). 2D Artists / UX Designers, preferably with Unity UI experience. Our menus still look pretty dull, and we don't like that. We also need concept art for characters and iconic game moments to define their look and feel. Coders. If you know your way around Unity and C#, there are lots of challenging things to be done. You will work closely together with the two programmers already on the team to get going quickly. Please drop me a message or contact info@three-eyed-games.com
    • By sveta_itseez3D
      itSeez3D, a leading developer of mobile 3d scanning software, announced today a new SDK for its automatic 3D avatar generation technology, Avatar SDK for Unity. The Avatar SDK for Unity is a robust plug-n-play toolset which enables developers and creatives to integrate realistic user-generated 3D avatars into their Unity-based applications. SDK users can allow players to create their own avatars in the application or integrate the SDK into their own production processes for character design and animation.
      “Virtual avatars have recently become increasingly popular, especially in sports games and social VR apps. With the advance of VR and AR, the demand to get humans into the digital world is only increasing”, said Victor Erukhimov, itSeez3D CEO. “Our new Avatar SDK for Unity makes it super-easy to bring the avatar technology into any Unity-based game or VR/AR experience. With the Avatar SDK for Unity now every developer can bring face scanning technology into their games and allow players to create their own personalized in-game avatars, making the gameplay much more exciting and immersive.”
      Key features of the Avatar SDK for Unity:
      Automatic generation of a color 3D face model from a single selfie photo in 5-10 seconds (!). Works best with selfies, but can be used with any portrait photo.
      Shape and texture of the head model are unique for each person, synthesized with a deep learning algorithm crafted by computer vision experts
      Head models support runtime blendshape facial animations (45 different expressions)
      Generated 3D heads include eyes, mouth, and teeth
      Algorithms synthesize 3D meshes in mid-poly resolution, ~12k vertices, and ~24k triangles
      Six predefined hairstyles with hair-recoloring feature (many more available on request)
      Avatar generation API can be used in design-time and in run-time, which means you can allow users to create their own avatars in your game
      Cloud version is cross-platform, and offline version currently works on PCs with 64-bit Windows (support for more platforms is coming soon)
      Well-documented samples showcasing the functionality.
       
      Availability
      The Avatar SDK for Unity is offered in two modes - “Cloud” and “Offline”. The “Cloud” version is available at http://avatarsdk.com/ and the “Offline” version is available by request at support@itseez3d.com.
      ###
      About itSeez3D
      At itSeez3D, we are working on the computer vision technology that turns mobile devices into powerful 3D scanners. itSeez3D has developed the world's first mobile 3D scanning application that allows to create high-resolution photorealistic 3D models of people's' faces, bodies and objects. The application is available for iOS and Windows OS mobile devices powered with 3D cameras. In 2016 the company introduced Avatar SDK that creates a realistic 3D model of a face from a single selfie photo. To learn more about itSeez3D scanning software and 3D avatar creation technology, please visit www.itseez3d.com and www.avatarsdk.com.

      View full story
    • By sveta_itseez3D
      itSeez3D, a leading developer of mobile 3d scanning software, announced today a new SDK for its automatic 3D avatar generation technology, Avatar SDK for Unity. The Avatar SDK for Unity is a robust plug-n-play toolset which enables developers and creatives to integrate realistic user-generated 3D avatars into their Unity-based applications. SDK users can allow players to create their own avatars in the application or integrate the SDK into their own production processes for character design and animation.
      “Virtual avatars have recently become increasingly popular, especially in sports games and social VR apps. With the advance of VR and AR, the demand to get humans into the digital world is only increasing”, said Victor Erukhimov, itSeez3D CEO. “Our new Avatar SDK for Unity makes it super-easy to bring the avatar technology into any Unity-based game or VR/AR experience. With the Avatar SDK for Unity now every developer can bring face scanning technology into their games and allow players to create their own personalized in-game avatars, making the gameplay much more exciting and immersive.”
      Key features of the Avatar SDK for Unity:
      Automatic generation of a color 3D face model from a single selfie photo in 5-10 seconds (!). Works best with selfies, but can be used with any portrait photo.
      Shape and texture of the head model are unique for each person, synthesized with a deep learning algorithm crafted by computer vision experts
      Head models support runtime blendshape facial animations (45 different expressions)
      Generated 3D heads include eyes, mouth, and teeth
      Algorithms synthesize 3D meshes in mid-poly resolution, ~12k vertices, and ~24k triangles
      Six predefined hairstyles with hair-recoloring feature (many more available on request)
      Avatar generation API can be used in design-time and in run-time, which means you can allow users to create their own avatars in your game
      Cloud version is cross-platform, and offline version currently works on PCs with 64-bit Windows (support for more platforms is coming soon)
      Well-documented samples showcasing the functionality.
       
      Availability
      The Avatar SDK for Unity is offered in two modes - “Cloud” and “Offline”. The “Cloud” version is available at http://avatarsdk.com/ and the “Offline” version is available by request at support@itseez3d.com.
      ###
      About itSeez3D
      At itSeez3D, we are working on the computer vision technology that turns mobile devices into powerful 3D scanners. itSeez3D has developed the world's first mobile 3D scanning application that allows to create high-resolution photorealistic 3D models of people's' faces, bodies and objects. The application is available for iOS and Windows OS mobile devices powered with 3D cameras. In 2016 the company introduced Avatar SDK that creates a realistic 3D model of a face from a single selfie photo. To learn more about itSeez3D scanning software and 3D avatar creation technology, please visit www.itseez3d.com and www.avatarsdk.com.
    • By Canislupus54
      I'm looking for programmers for an rpg I want to make. If you're wondering what "semi-turn based" means, it means that you take turns, but instead of a rigid back and forth like Pokemon, a timer determines when you can act, a sort of modernization of the classic Final Fantasy Active Time Battle system. Right now, I'm looking for programmers to create a prototype of both the combat system and the movement outside of combat. Preferably for Unity C#. Concept artists, particularly for characters, and writers to help me flesh out the character and story aspects, would also be helpful.
      Here's a concept doc to fully explain things: https://docs.google.com/document/d/1ObDMAUWsndSAJ1EpQGRDxtR8Xl9xPotx89OZ0sgRaIw/edit?usp=sharing
      If you can fill another role and are interested, feel free to let me know as well.
      At the moment, this is purely a hobby project, with no payment planned. If we produce something we feel we can release, then of course we'll work out something for compensation. But, again, don't join this project counting on payment.
      If you're interested, contact me on here, or at jordestoj@yahoo.com . Thanks.
  • Popular Now