• Announcements

    • khawk

      Download the Game Design and Indie Game Marketing Freebook   07/19/17

      GameDev.net and CRC Press have teamed up to bring a free ebook of content curated from top titles published by CRC Press. The freebook, Practices of Game Design & Indie Game Marketing, includes chapters from The Art of Game Design: A Book of Lenses, A Practical Guide to Indie Game Marketing, and An Architectural Approach to Level Design. The GameDev.net FreeBook is relevant to game designers, developers, and those interested in learning more about the challenges in game development. We know game development can be a tough discipline and business, so we picked several chapters from CRC Press titles that we thought would be of interest to you, the GameDev.net audience, in your journey to design, develop, and market your next game. The free ebook is available through CRC Press by clicking here. The Curated Books The Art of Game Design: A Book of Lenses, Second Edition, by Jesse Schell Presents 100+ sets of questions, or different lenses, for viewing a game’s design, encompassing diverse fields such as psychology, architecture, music, film, software engineering, theme park design, mathematics, anthropology, and more. Written by one of the world's top game designers, this book describes the deepest and most fundamental principles of game design, demonstrating how tactics used in board, card, and athletic games also work in video games. It provides practical instruction on creating world-class games that will be played again and again. View it here. A Practical Guide to Indie Game Marketing, by Joel Dreskin Marketing is an essential but too frequently overlooked or minimized component of the release plan for indie games. A Practical Guide to Indie Game Marketing provides you with the tools needed to build visibility and sell your indie games. With special focus on those developers with small budgets and limited staff and resources, this book is packed with tangible recommendations and techniques that you can put to use immediately. As a seasoned professional of the indie game arena, author Joel Dreskin gives you insight into practical, real-world experiences of marketing numerous successful games and also provides stories of the failures. View it here. An Architectural Approach to Level Design This is one of the first books to integrate architectural and spatial design theory with the field of level design. The book presents architectural techniques and theories for level designers to use in their own work. It connects architecture and level design in different ways that address the practical elements of how designers construct space and the experiential elements of how and why humans interact with this space. Throughout the text, readers learn skills for spatial layout, evoking emotion through gamespaces, and creating better levels through architectural theory. View it here. Learn more and download the ebook by clicking here. Did you know? GameDev.net and CRC Press also recently teamed up to bring GDNet+ Members up to a 20% discount on all CRC Press books. Learn more about this and other benefits here.
  • entries
    94
  • comments
    271
  • views
    150226

C# for scripting - runtime compilation

Sign in to follow this  
Followers 0
evanofsky

1616 views

I set out to add scripting support to Project Lemma the other day. End result: I can recompile C# scripts on the fly and cache the bytecode in DLLs. The best part: there's no special binding code, and no performance hit.

There are a lot of .NET scripting solutions out there. Here's a few I found in my research:[size="2"][size="2"][size="2"][size="2"]

  • IronPython. Fully dynamic, kinda slow. Requires marshalling of some kind between the script world and .NET.
  • CSScript. Very well supported, includes Visual Studio extensions and shell extensions. Compiles C# to bytecode at runtime, with caching. Scripts cannot be changed once loaded.
  • Lua. The industry standard in scripting. From what I understand, a little challenging to get working with .NET.
    I decided to try out C# runtime compilation. Really, C# is the best scripting language I could ask for. If I succeeded, I could keep writing the same code I've been writing, but I could recompile it and see it in action with a keystroke instead of restarting the game!

    [subheading]Setup[/subheading]
    I started by writing a ScriptBase class that every script would inherit from.

    public class ScriptBase
    {
    public static Main main;

    protected static Entity get(string id)
    {
    return ScriptBase.main.GetByID(id);
    }
    }

    "Main" is my main game class. The "get" function is a utility function I wanted to provide for the scripting environment. Using the ScriptBase class, I can add utility functions that can speed up the scripting process.

    Next, I wrote a prefix and postfix for the script files. This will help the scripts actually look like scripts, i.e. not object oriented, pretty much just a list of statements to execute in order.

    private const string scriptPrefix =
    @"
    using System;
    using Microsoft.Xna.Framework;
    // ...

    namespace Lemma.Scripts
    {
    public class Script : ScriptBase
    {
    public static void Run()
    {
    ";

    private const string scriptPostfix =
    @"
    }
    }
    }
    ";

    The idea is: compile the assembly, reflect it and get the Script type, then call the static Run function on it. Like so:

    Type t = assembly.GetType("Lemma.Scripts.Script");

    t.GetField("main", BindingFlags.Static | BindingFlags.Public | BindingFlags.FlattenHierarchy)
    .SetValue(null, this.main);

    this.scriptMethod = t.GetMethod("Run", BindingFlags.Static | BindingFlags.Public);
    this.scriptMethod.Invoke(null, null);

    [subheading]Compile some scripts already![/subheading]
    This code isn't particularly interesting, but I'll provide it for reference:

    string scriptPath = ...;
    string binaryPath = ...;

    try
    {
    Assembly assembly = null;

    using (Stream stream = TitleContainer.OpenStream(scriptPath))
    using (TextReader reader = new StreamReader(stream))
    {
    CodeDomProvider provider = CodeDomProvider.CreateProvider("CSharp");

    CompilerParameters cp = new CompilerParameters
    {
    GenerateExecutable = false,
    GenerateInMemory = false,
    TreatWarningsAsErrors = false,
    OutputAssembly = binaryPath
    };

    // Add references to all the assemblies we might need.
    Assembly executingAssembly = Assembly.GetExecutingAssembly();
    cp.ReferencedAssemblies.Add(executingAssembly.Location);
    foreach (AssemblyName assemblyName in executingAssembly.GetReferencedAssemblies())
    cp.ReferencedAssemblies.Add(Assembly.Load(assemblyName).Location);

    // Invoke compilation of the source file.
    CompilerResults cr = provider.CompileAssemblyFromSource(cp, Script.scriptPrefix + reader.ReadToEnd() + Script.scriptPostfix);

    if (cr.Errors.Count > 0)
    {
    // Display compilation errors.
    StringBuilder builder = new StringBuilder();
    foreach (CompilerError ce in cr.Errors)
    {
    builder.Append(ce.ToString());
    builder.Append("\n");
    }
    this.Errors.Value = builder.ToString();
    }
    else
    assembly = cr.CompiledAssembly;
    }
    }
    catch (Exception e)
    {
    this.Errors.Value = e.ToString();
    }

    The goal here is to compile the script to a DLL so the next time the game runs, I could check if it already exists and just load the DLL instead of recompiling. You can do this with the CompilerParameters object, by setting GenerateInMemory to false and OutputAssembly to the path of the DLL you want to save.

    [subheading]Recompiling scripts on the fly[/subheading]
    Now the problem is, once we load the DLL, the .NET runtime locks the file until the program exits. There's no way to unload the library and unlock the file, unless we put it in a different AppDomain, which means we have to marshal data back and forth between the script and game, which defeats the purpose.

    Since one of our requirements is being able to recompile the script without restarting the game, we need to find a way to load the DLL without locking the file. Enter shadow copying. You can tell the .NET runtime to make a copy of every assembly it loads, and load the "shadow copy" instead of the real file. That leaves the original DLL free for us to rewrite. Great, so let's just enable shadow copying and we're done.

    Unfortunately, life is never that easy. You can't enable shadow copying for an AppDomain once the domain has been created. You have to create a new AppDomain with the settings you want. Solution: create a tiny launcher executable that creates an AppDomain with shadow copying enabled, and then executes our game in that AppDomain. Here's the code:

    public static void Main(string[] args)
    {
    string baseDirectory = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);

    AppDomainSetup setup = new AppDomainSetup();
    setup.ShadowCopyFiles = "true";
    setup.ApplicationBase = baseDirectory;
    setup.PrivateBinPath = baseDirectory;

    AppDomain domain = AppDomain.CreateDomain("", AppDomain.CurrentDomain.Evidence, setup);
    domain.ExecuteAssembly(Path.Combine(baseDirectory, "Lemma.exe"), args);
    }

    Note: the AppDomainSetup.ShadowCopyDirectories property lets you provide a list of directories to limit the use of shadow copying. I tried to use it to specify that only the script DLLs be shadow copied, but it didn't work. YMMV.

    Okay, so I fired up my game, compiled the script, went to delete the compiled DLL, annnnnd... still locked! What's going on?

    I did some reflecting and found out the C# runtime compiler is kinda janky. It isn't, like I assumed, a .NET wrapper around some kind of compiler library. Instead, it actually invokes the C# compiler executable, then loads the resulting DLL. If you specify GenerateInMemory = true, it invokes the compiler with a temporary DLL file path and loads the temp file into memory.

    Furthermore, it doesn't shadow copy the DLL when it loads it, even if the AppDomain is configured to do so. I'm guessing this is because it uses a different Assembly.Load function... don't quote me but I believe Assembly.LoadFile does not shadow copy files, while Assembly.LoadFrom does.

    Anyway, to solve this, we can take advantage of the compiler's quirky nature here by not specifying an output assembly path. That will cause the compiler to generate a temporary file path for the output assembly. So it will still lock the DLL, but we don't care, because it's a temp file. Luckily, we can still read the DLL even though it's locked, so we can copy it to the path we originally wanted for later use.

    Here's the modifications to our original compilation code:
    CompilerParameters cp = new CompilerParameters
    {
    GenerateExecutable = false,
    GenerateInMemory = false,
    TreatWarningsAsErrors = false,
    };

    // ...


    assembly = cr.CompiledAssembly;
    File.Copy(cp.OutputAssembly, binaryPath, true);

    The next time we load the script, we'll see the DLL at binaryPath and load it. But it will be automatically shadow copied. Mission accomplished!

    [subheading]Conclusion[/subheading]
    One disadvantage to this approach is that when you recompile a script, the old assembly remains in memory, and the temp DLL file will remain locked until the game exits. Like I said, there's no way to unload an assembly from a running AppDomain, and we don't want to put the scripts in another AppDomain. Luckily most script assemblies should be pretty small, and we'll only be recompiling scripts in the editor, not the final game. I can live with that.

    I'm still in the middle of all this, but I thought I'd share what I've learned about .NET in the meantime. Might be useful if you're thinking about doing something similar. Here's two StackOverflow threads that helped me through this process:


0
Sign in to follow this  
Followers 0


5 Comments


Can you go into a little more detail about what your scripts would look like, perhaps posting an example or two? The terms "prefix" and "postfix" seem a little confusing in this context.
1

Share this comment


Link to comment
Sure. The prefix and postfix are just strings that I tack on at the beggining and end of each script.

An example script might be something like this:

[code]get("player1").Get<Transform>().Position.Value = new Vector3(100, 0, 50);[/code]
Obviously this isn't a complete C# source file. It needs "using" statements, and it needs to be put in a function, inside a class, inside a namespace. That's what the prefix and postfix do, so I don't have to write that stuff in every script. I can still create functions and such via closures.

tl;dr: It's just for convenience.
0

Share this comment


Link to comment
Lua is really easy to get working with C# if you use the library [url="http://code.google.com/p/luainterface/"]LuaInterface[/url]. It does suffer from the lack of really good documentation and I haven't seen a release in quite a while. However, I've been using it for a few years now and the only modification I had to make was the proper handling of params arguments (which the code needed to do this can be found in the bug tracker.)
1

Share this comment


Link to comment
Interesting idea. Does the rest of your game use C# or do you have some C++ stuff mixed in?

My approach is to build a game using a bunch of C++ libraries (graphics, sound, etc) with wrappers around them and put the game code together with C# (but not XNA). It works really well once you get the boilerplate code done. The performance is very promising too. I don't have the advantage of on the fly scripting but the build time and dev time is significantly better than C++ as you'd already know.

Anyway, I'll be following your posts. Interesting stuff.
1

Share this comment


Link to comment
@Programmer16: That sounds promising... another developer that used my approach here said that the memory leak issue I mentioned got to be a big problem. If that happens to me too, I'll definitely look into LuaInterface.

@zarfius: It's all C#, which is what makes it so convenient. Your approach sounds a lot like Panda3D. All their high performance code is C++, but everything is exposed to Python. Best of both worlds!
0

Share this comment


Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now