• Advertisement

Archived

This topic is now archived and is closed to further replies.

Scripting vs Compiled code (not what you think)

This topic is 4971 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Yeah, this probably does belong in the scripting forum, but whatever. Anyway, here''s the deal. Right now I''m looking at the topic of scripting. I''m only just starting out, but I''m pretty sure I get the concept. You compile a script into your own "machine code" at load time, and run it in your virtual machine (essentially an emulator for an instruction set that never existed in hardware). Ok, now the technical details aside, I have a question. Why don''t we just compile those scripts into actual native machine code, and execute it directly? Ok, I understand how risky this sounds. Afterall, a virtual machine is a protected environment we can control. When you remove that safeguard, someone could essentially hijak our program, etc etc. The fact is I''d probably never do this, I just want to get some feedback on the idea. I want someone to convince me why not to. On the technical side, at this point in time I can''t think of a reason why it couldn''t be done. It''d be easy enough to compile it for a specific location in your memory space, and since you''re doing the compilation, you can control the linking, so you can dictate what functions it''s allowed to call, and map back those function calls to the actual locations those functions are located at in your compiled program. Since this is still being done at load time, you can still modify both the scripts and your main program seperately, maintaining thier independence. There are two main advantages I see to doing this. -Significantly faster execution of your scripts, since you''re executing the code, not interpreting it. For games with some hefty AI, this could result in a noticeable performance boost. -No need to develop a virtual machine at all really. All you end up with is a loose wrapper over your scripts, if anything at all. I want some other people to talk through the disadvantages with (and possibly, any more advantages I haven''t thought of yet). Also, I want to know if anyone else has thought of/tried this before. Anyway, opinions?

Share this post


Link to post
Share on other sites
Advertisement
A lot of scripting languages do compile to machine code. Ultimately, though, most of the runtime overhead of a scripting language is not interpreting the bytecode, but the support systems such as dynamic types and garbage collection. These are facilities that are already running natively, so there''s no potential speedup there. The performance does improve, but the resultant code is still nowhere near as fast as natively compiled C or C++ code.


"Sneftel is correct, if rather vulgar." --Flarelocke

Share this post


Link to post
Share on other sites
I didn''t realise there were any implementations today that did this. Can you give me any examples? I''m not doubting you, I''d just like to look into this further.

Share this post


Link to post
Share on other sites
quote:
-No need to develop a virtual machine at all really. All you end up with is a loose wrapper over your scripts, if anything at all.

but, you would have to develop a compiler instead, which is at least as difficult and time-consuming.

this is not to say i think it is a bad idea

Share this post


Link to post
Share on other sites
quote:
but, you would have to develop a compiler instead, which is at least as difficult and time-consuming.

Yeah, but if you''re going to compile it from text into some kind of bytecode, you really need a compiler anyway. The only difference is that there''s probably more resources you can refer to on regular old X86.

Share this post


Link to post
Share on other sites
Hmm... Scheme, I believe, has a native-code compiler, as does Python... Also, there are Java JIT compilers, which are pretty much the same. I don''t remember whether there''s a native Perl compiler.


"Sneftel is correct, if rather vulgar." --Flarelocke

Share this post


Link to post
Share on other sites
Nemesis2k2:

i have implemented scripting similar to what you describe.

there is a cool imbedded assembler library for c++ called 'softwire' that has function calls that will translate the assembly to machine code, allowing your script compiler to just output assembly and then use softwire to create the actual native code. (this can all be done on the fly in memory without creating files)

The challenges:

Its harder to compile to assembly and keep up with register usage etc in the compiler than it is just writing to your own virtual machine.

Writing a source level debugger is MUCH MUCH harder when dealing with native code than with a virtual machine. Although you can cheat and have a debug function called for each line as the native code is run, that doesn't solve the problem of inspecting variables.

hope this helps


[edited by - imbro on June 10, 2004 12:51:50 AM]

Share this post


Link to post
Share on other sites
quote:
Original post by imbro

Its harder to compile to assembly and keep up with register usage etc in the compiler than it is just writing to your own virtual machine.

[edited by - imbro on June 10, 2004 12:09:49 AM]


Not really, I wrote a script language/virtual machine using Softwire and translated the virtual machine instructions directly to the x86 architecture. It was not difficult at all, but the language is very primitive like a tiny C compiler, nothing like Python etc.

Basically I have a stack machine with 2 registers A,B. As you guessed in expressions I map A->eax and B->ebx. Occasionally I use the hardware stack when it is needed to store a temporary op.

All in all it runs fairly fast.

Share this post


Link to post
Share on other sites
This softwire thing is pretty neat. It takes care of most of the things I was worried about. Scripting will be playing a major part in my engine later on, so I think I''ll probably be coming back to this in the future.

quote:
Writing a source level debugger is MUCH MUCH harder when dealing with native code than with a virtual machine. Although you can cheat and have a debug function called for each line as the native code is run, that doesn''t solve the problem of inspecting variables.

An easy way of inspecting variables is a luxury anyway. I''ve done development for some old systems where an interrupt after each command was the only option for debugging, so I think I''ll make do.

quote:
This is similiar to a Just In Time compiler.

After a bit of research, yes. I''d heard that term before but I didn''t know what it meant. I just got curious about this because everything I''d heard about in terms of scripting for games talked about interpreting it, and I couldn''t see a good reason why you''d bother with the potential performance difference.


Anyway, thanks for the responces. Good to know the idea was sound afterall.

Share this post


Link to post
Share on other sites
I use an evil mix of C++ code and a sort of scripting for use in my AI engine. Everything is compiled with the cygwin (gcc) compiler and thrown into DLLs so its fast and can be changed run-time. Pretty sweet, but I can only allow this because the source is trusted (our own developers work on the code and it all runs on a tightly buttoned up server).
In the future I DO plan to extend the compiler so that the need for C++ coding will be erraticated, but for now I really dont see the need for such tight security.

Emil Johansen
- SMMOG AI designer
http://smmog.com

Share this post


Link to post
Share on other sites
quote:
Original post by Sneftel
but the resultant code is still nowhere near as fast as natively compiled C or C++ code.



Lua5 is awfully close in this shootout.
http://dada.perl.it/shootout/craps.html

The main question you should be asking:
Is your time more important than the computers? If I finish my code in 25% of the time, I have 75% more time to optimize and determine which parts are actually slow.

Lets say your script code is 20% slower than native code is. Computers today are 3.0 GigaHertz, which means your scripts are effectively running at 2.4 Gigahertz. Which means you get MUCH higher productivity and the speed of C 18 months ago.

Compiling is just one optimization. Algorithmic changes, lookup tables and memoization are much more effective optimization techniques, than compiling, IMO.

Share this post


Link to post
Share on other sites
A Just In Time compiler takes bytecode, then compiles it into machine code for each different computer the first the program is run.

Share this post


Link to post
Share on other sites
It''s quite possible for a game to have a VM just like Java or CLR, that actually does JIT compiling - but it''s mostly unecessary. You might as well just make your own programming language. Scripting is supposed to be there to make certain things simpler, not to give you a wholly new language to develop your game in.

I think Unreal is probably the best example of a well-implemented scripting engine. It doesn''t touch any of the low level stuff, and it compiles into bytecode which runs at a "reasonable" speed (10-15% the speed of the C++ code), and it keeps designers from having to worry about stuff that they shouldn''t need to worry about.

Share this post


Link to post
Share on other sites

  • Advertisement