Jump to content

  • Log In with Google      Sign In   
  • Create Account

Compiling and replacing classes at runtime in C#


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
13 replies to this topic

#1 Structural   Members   -  Reputation: 328

Like
0Likes
Like

Posted 20 May 2008 - 07:11 AM

I got this idea in my head two minutes ago when I read this thread about creating a scripting language and C# was mentioned. At some point I will also need a scripting language, and I haven't given it that much thought (other than LUA is probably going to be the best candidate). So it is possible to compile and execute C# code at runtime. So from that point of view C# would be a possible candidate for a scripting language, if not for one important feature that I can't find the answer to right now: Is it possible to RECOMPILE and replace running code with C#/.NET? A'la "edit and continue". This is an important feature when tweaking variables in your game. Does anyone have experience with this? If so, how do you do it, and what are your thoughts about it?

Sponsor:

#2 Mike.Popoloski   Crossbones+   -  Reputation: 2931

Like
0Likes
Like

Posted 20 May 2008 - 07:30 AM

I'm not sure what you mean, tweaking variables. Variables can be changed at runtime, without recompiling anything.

Anyway, you can do just about anything you can think of in C# at runtime, including dynamically changing methods, but it could turn out to be rather difficult, depending on what you are doing and whether you need to delve into emitting IL opcodes.
Mike Popoloski | Journal | SlimDX

#3 Telastyn   Crossbones+   -  Reputation: 3730

Like
0Likes
Like

Posted 20 May 2008 - 08:40 AM

I'd guess so, but more importantly, why?

You can assign and generally manipulate arbitrary variables at runtime via reflection. Combine the runtime types with reflection and you can assign the runtime type to an existing variable. Even generic types can be bound via reflection.

That said, your custom code/type will still have to have the type of the variable you're assigning to. If you want true dynamic types common in most scripting languages, then they might be more applicable...



#4 Structural   Members   -  Reputation: 328

Like
0Likes
Like

Posted 20 May 2008 - 07:11 PM

Ok, so the tweaking of variables may not be the best example and can be achieved in an other ways.
Though I feel there are advantages by putting variables and constants in a "C# script" and being able to edit and reload the script at run time. For example, let's say you have a variable named "run speed". Yes, I could use reflection to change that variable, but that means I would somehow have to have an interface in the game that allows me to change that variable. This interface to change variables would be rather cryptic, depending on the amount of variables you might want to change and the flexibility you'd want. It would be much more convenient to edit a script file and have it reloaded automatically (using file listeners and whatnot).


Also, what if you want to change the code inside a method? An AI routine for example. At run time.


Actually, the asking "WHY" I would want to change methods and variables at run time is the same as asking "why would you want scripting in the first place": flexibility. I'm just wondering if C# is a good candidate as a scripting language.

Quote:
Anyway, you can do just about anything you can think of in C# at runtime, including dynamically changing methods

Could you give me a pointer to where I can find how to do this? A google with the keywords "dynamically changing methods" doesn't turn up anything that seems usefull.


Quote:
That said, your custom code/type will still have to have the type of the variable you're assigning to. If you want true dynamic types common in most scripting languages, then they might be more applicable...

Now you mention this, this might indeed be a drawback. How important is having generic types in a game scripting language? I don't have enough experience with game related scripting to have an answer to that.
Also, what will happen if you change a type's internals and you already have instances of that type in your application?
I'm going to have to spike this when I have some time.

#5 Rainweaver   Members   -  Reputation: 220

Like
0Likes
Like

Posted 20 May 2008 - 08:01 PM

CS Script
A similar request in the page comments

I've been working with dynamic method generation some time ago, and I can tell you from my experience that it's not an easy task to undertake. It might take you precious time.

Look for "C# dynamic IL generation", or something along those line. You'll likely have to understand how IL works too, as mentioned before. It's an interesting trip, but if you can use Lua, don't go there yet.

My 2 cents.

#6 Structural   Members   -  Reputation: 328

Like
0Likes
Like

Posted 22 May 2008 - 07:23 PM

Quote:
Original post by Rainweaver
CS Script
A similar request in the page comments

I've been working with dynamic method generation some time ago, and I can tell you from my experience that it's not an easy task to undertake. It might take you precious time.

Look for "C# dynamic IL generation", or something along those line. You'll likely have to understand how IL works too, as mentioned before. It's an interesting trip, but if you can use Lua, don't go there yet.

My 2 cents.


Thanks for the link. I took a peek at the CSScript code and it seems it's indeed using the .NET framework's compiler to compile the code and load assemblies. It does some parsing magic to also load referenced assemblies.
It doesn't seem to have edit-and-continue functionality though.

There IS a way to get edit-and-continue functionality I think. Basically you have to compile your code and load the compiled assembly in a new AppDomain. From there you should be able to execute it. Crossing AppDomain boundaries and creating objects in another AppDomain requires a specific interface. So invoking script will not look as if it was normal C#. Which in my opinion is a good thing. Makes it explicit you are doing something special.

If you want to reload the script you would have to destroy the AppDomain in which your "script" is running, recompile the code again in a new assembly, and then create yet another AppDomain for the new script.
This part should not be that hard to do.

The hard part (ie: the part I haven't really figured out yet) is that instances of script objects will exist in the script's AppDomain. So if you edit-and-continue, and thus destroy the old AppDomain those objects are also destroyed. You somehow have to recreate the objects from the old AppDomain in the new one.
Now, I did read that if you create an object in another AppDomain you get a proxy object if you use CreateInstanceAndUnwrap.
Using reflection you MIGHT be able to extract the state of the object, and in that way carefully copy the state to the new object. Whether this works or not depends if you can reflect the proxy object that you got from the other AppDomain.

I still have to spike if this works. I know nearly nothing about AppDomains, so spiking this will be a good exercise.

#7 Rainweaver   Members   -  Reputation: 220

Like
0Likes
Like

Posted 23 May 2008 - 01:12 AM

Interesting links. I'm about to work with AppDomains too.

Good luck with your implementation and let us know!

#8 RobTheBloke   Crossbones+   -  Reputation: 2341

Like
0Likes
Like

Posted 23 May 2008 - 01:52 AM

Quote:
Original post by Structural
Yes, I could use reflection to change that variable, but that means I would somehow have to have an interface in the game that allows me to change that variable.


Which is what reflection gives you. Alternatively expose all properties properly, and use the PropertyGrid control on the object (or your own reflection).

Quote:
Original post by Structural
This interface to change variables would be rather cryptic, depending on the amount of variables you might want to change and the flexibility you'd want. It would be much more convenient to edit a script file and have it reloaded automatically (using file listeners and whatnot).


Still sounds like you want to use reflection. Not sure why you are worrying so much about it?

Quote:
Original post by Structural
Also, what if you want to change the code inside a method? An AI routine for example. At run time.


the same as any other scripting language, you have to re-compile it. C# scripting does have an overhead when you re-compile, so either farm it onto another thread, or do all compilation when you load the level (which is the most common usage of a scripting language anyway).

Quote:
Original post by Structural
Actually, the asking "WHY" I would want to change methods and variables at run time is the same as asking "why would you want scripting in the first place": flexibility. I'm just wondering if C# is a good candidate as a scripting language.


Yes, it's a good candidate for a scripting language.

Quote:
Original post by Structural
Could you give me a pointer to where I can find how to do this? A google with the keywords "dynamically changing methods" doesn't turn up anything that seems usefull.


example src
some description about the code


Quote:
That said, your custom code/type will still have to have the type of the variable you're assigning to. If you want true dynamic types common in most scripting languages, then they might be more applicable...


!????? object ??????!.

Quote:
Now you mention this, this might indeed be a drawback. How important is having generic types in a game scripting language? I don't have enough experience with game related scripting to have an answer to that.


!????? object ??????!.

Quote:
Also, what will happen if you change a type's internals and you already have instances of that type in your application?


Nothing. The instances will retain the old type info, any new instances will use the new type info. You'd have to manually release the old instances and re-create them for it to work. (unless you were to seperate the data and process sections of an object definition, i.e. the data gets retained, and the functional part of the class gets re-defined).

#9 RobTheBloke   Crossbones+   -  Reputation: 2341

Like
0Likes
Like

Posted 23 May 2008 - 01:57 AM

Quote:
Original post by Structural
There IS a way to get edit-and-continue functionality I think. Basically you have to compile your code and load the compiled assembly in a new AppDomain. From there you should be able to execute it. Crossing AppDomain boundaries and creating objects in another AppDomain requires a specific interface. So invoking script will not look as if it was normal C#. Which in my opinion is a good thing. Makes it explicit you are doing something special.


It's trivial to make C# script and C# code identical, the example link i posted does just that (i.e. spot the difference between the CS plugin code, vs the CS script code). If you choose to invoke your scripts differently to say, a plugin, then that's your choice. Personally I prefer keeping script and C# code identical, since it lends itself to rapid prototyping via scripts, then move the script into the main codebase and compile in with the rest when done.

Quote:
The hard part (ie: the part I haven't really figured out yet) is that instances of script objects will exist in the script's AppDomain. So if you edit-and-continue, and thus destroy the old AppDomain those objects are also destroyed. You somehow have to recreate the objects from the old AppDomain in the new one.


Which is why i don't like that method.

Quote:
Using reflection you WILL be able to extract the state of the object, and in that way carefully copy the state to the new object. Whether this works or not depends if you can reflect the proxy object that you got from the other AppDomain.


fixed.

#10 Umbrae   Members   -  Reputation: 308

Like
0Likes
Like

Posted 23 May 2008 - 02:24 AM

I asked a very similar question: Edit and continue.
A resource I found useful was: Shadow Copying Assemblies.

#11 Structural   Members   -  Reputation: 328

Like
0Likes
Like

Posted 26 May 2008 - 06:40 PM

Quote:
Original post by Umbrae
I asked a very similar question: Edit and continue.
A resource I found useful was: Shadow Copying Assemblies.


I was wondering how you used the shadowcopying.

I did a spike of loading the script assembly in a separate AppDomain. This separate AppDomain has shadowcopying turned on. This part works just fine. I can write to the generated assembly again.
The problem however is that the calling assembly/application also wants to load the script assembly and thus locks the assembly file. The reason it does this is to provide reflection over the objects in the new AppDomain. That makes perfect sense, but it means that the script assembly is loaded in the script AppDomain as well as in the main AppDomain, so I can't unload it anymore from the main AppDomain.

This is turning out to be quite a pain. Compiling the script and loading it is easy as pie. Having a compiler built in the .NET framework is great.
But unloading/reloading assemblies is quite a hassle because the calling assembly also loads the script assembly.

#12 Umbrae   Members   -  Reputation: 308

Like
0Likes
Like

Posted 27 May 2008 - 01:29 AM

Quote:
Original post by Structural
I was wondering how you used the shadowcopying..

I didn't get far enough to use it, so I guess I would be in the same boat as you.

If you do find out (and likewise if I find out) let everyone know, I'm sure it's useful information.

#13 Structural   Members   -  Reputation: 328

Like
0Likes
Like

Posted 28 May 2008 - 05:36 AM

Quote:
Original post by Umbrae
Quote:
Original post by Structural
I was wondering how you used the shadowcopying..

I didn't get far enough to use it, so I guess I would be in the same boat as you.

If you do find out (and likewise if I find out) let everyone know, I'm sure it's useful information.


I found a hint: scriptAppDomain.Load() will indeed force the assembly to be loaded in the current appdomain.

You could create a shared "ScriptEngine" assembly, and invoke Assembly.Load() through a remote call on the script appDomain. The script engine assembly will be locked, but that won't be an issue as the actual script will be in yet another assembly.

I think you could use Assembly.ReflectionOnlyLoad() to get reflection info hopefully without locking the assembly.
One thing that is still a blank more or less is how easy it'll be to invoke methods and getting members with only having reflection info.


So, to recap how to get around the locking of the assembly:

- create "Script Engine" assembly
- create script appDomain
- load script engine assembly in script's appdomain
- invoke a method in your script engine assembly and in the script appDomain to load your actual script assembly
- use Assembly.ReflectionOnlyLoad() to get reflection info of the script assembly (UNCERTAIN if this leaves the assembly file unlocked, MSDN doesn't state)
- somehow invoke methods and get members in the script appDomain

I'm going to try out the last two items. If that works I believe all bases are covered for using C# as a scripting language (the way I (think I) want it to work at least).

[edit]
Assembly.ReflectionOnlyLoad() DOES lock the file... buggery! A quick google seems to indicate there is no way around it.
This also means that anything you try to reflect the object will result in an attempt to load the assembly (reflection only or not) in the main appDomain. Even myScriptObject.GetType() from the main appDomain will.

One way to get around this may be to give the script engine an interface to invoke methods and get/set member variables and serialize/deserialze objects. But I don't feel it is viable to build all that for my personal project.

And another thing just hit me... even IF I can invoke script from the game, I still haven't figured out how to invoke game methods from script.
This poses an entirely new problem I think.

Right... I'm going to put this to rest now. This whole appDomain boundary stuff is giving me a headache.

If you figure it out Umbrae then let me know. I'm still hoping it is possible.
[/edit]

[Edited by - Structural on May 28, 2008 12:36:50 PM]

#14 Ens   Members   -  Reputation: 122

Like
0Likes
Like

Posted 14 June 2008 - 08:40 AM

Check out the "CLR Managed Debugger (mdbg) Sample". It might allow the edit and continue for your scripting. Also i have used IronPython rather successfully for a scripting language (linked to from the link below)

http://www.microsoft.com/downloads/details.aspx?FamilyID=38449a42-6b7a-4e28-80ce-c55645ab1310&displaylang=en


(edit-add:)
IronPython provided a very seamless and simple interface for scripting. No mucking around with loading assemblies or emitting IL. I could probably put together a working sample app with less than 15 lines of code to introduce scripting.

I created an entire scripting framework to handle and manage any number of different scripting engines simultaneously. The framework code weight is a meager 210 lines. The IronPython interface adds only 120 more AND of this only about 14 lines are actually dealing with IronPython - the rest of them are interactions with the framework.

Needless to say i have been very impressed with IronPython ;)

[Edited by - Ens on June 14, 2008 3:40:32 PM]




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS