Jump to content
  • Advertisement
Sign in to follow this  
LackOfGrace

Send code to compiler

This topic is 4075 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi! i just got this crazy idea. for my 2d engine i was planning to have a script language to handle object updates and such, but discarded the idea because it would be to slow. now, i where thinking about creating a developer program for my engine. here is the idea. i could let the user create object types and write code for special events (like update , collision etc ) and then the app will stich together the code and send it to a compiler, wich will compile the code to machine code. it would remove slowness from having a script language, but it would grant the engine a better development solution. so, is there anyone that has created a compiler or used one without the overlaying IDE. i would need to know how to send header files and how to send libs for linking. /Victor

Share this post


Link to post
Share on other sites
Advertisement
Are you sure a scripting solution is too slow for your needs or are you just assuming? Think about that, and if you still believe it's too slow then you need to evaluate how you plan to use scripting to control behavior.

Scripting isn't intended for low-level implimentation of behaviors, its intended more for high-level behavior and response. The example I always use is this: A Script might say "Go to this position" or "follow this character" but you don't impliment the path-finding (A* or what-have-you) in the scripting language. 99% of the time if its too heavy for scripting, it wasn't a good match for scripting in the first place.


Now, if you truly do have a need for customizable plug-ins with native-code speed, you'll probably be best off looking into a Plug-in architecture using DLLs, rather than attempting to generate native code on the fly. The DLLs will have to be compiled seperately, so its not a run-time process like scripting, but you can provide a small "SDK" of sorts for your application. This SDK might contain something like an "IEntityController" abstract base class, which users can derive from and compile into a DLL which the application can load, along with their standard scripts.

Attempting to generate code at run-time is generally a bad idea unless you're an expert programmer. Even assuming you use a stable, existing library to generate the code there are still many difficulties. The primary difficulty is that your IDE's debugger will not be able to help you at all for run-time generated code. At best this means you are left to mentally evaluate and debug any flaws. It gets worse though because, eventually, harder-to-track bugs will probably require you to look at, and understand, the machine-code it spit out.

Share this post


Link to post
Share on other sites
Quote:
Original post by LackOfGrace
so, is there anyone that has created a compiler or used one without the overlaying IDE.


Most compilers are available without any sort of IDE. MSVC++ 6.0, 7.0, 7.1, and 8.0, along with gcc and borland (plus probably several others) are all usable just from the command line.

I'd also like to reiterate what ravyne2001 said: Scripts should be used to control high level behaviors, and have you actually written them and tested their performance before judging them as too slow?

Share this post


Link to post
Share on other sites
You don't "send" header files, they're included into the source files by the preprocessor. There's two stages to creating an exe or dll:

  1. Compiling
    The compiler compiles one module (.cpp file) at a time, and the preprocessor includes any header files needed into the file. By the time it gets to the compiler, it's just one large file with all the preprocessor definitions expanded. The compiler generates an intermediate file for this module (a .obj usually).

  2. Linking
    The linker comes next, after every source file has been compiled to object code, and links them together into one chunk of code along with any .lib files and generates the rest of the required information (header, import / export table, and so on)


You can have a look at the build log once you compile a program with Visual Studio 2005 if you want to see what command line it's generating for cl.exe (the compiler) and link.exe (the linker).

However, using machine code for scripting isn't usually a good idea as it opens up all sorts of security holes and completely removes the safety of scripting languages (If a script crashes, it doesn't usually bring down the host program).

Share this post


Link to post
Share on other sites
as i see it there are 3 possible ways of doing what i want.

1. DLL
let the user create a dll that will be run by the main application

pros:
fast

cons:
wont have extensive access to engine
(could be a good thing, though it would take some time to release features that the user would like)

2. Script

pros:
simple for the user

const:
writing my own takes time (could use others)
slow
would need an extensive database with commands

3. Compiled code

pros:
total access to engine
best speed

const:
need to write a interface for programming (will be doing this anyway since the user would need to create material files and such)

so, i really think that creating a ide for the engine would be really nice.
and event driven programming (the user gets to implemnt only update functions etc) is a proven concept (actionscript)


and i think it would be really funny to see if it was doable in a good way.


anyone have any articles or should i resort to command line references?



Share this post


Link to post
Share on other sites
Quote:
Original post by LackOfGrace
as i see it there are 3 possible ways of doing what i want.

1. DLL
...

cons:
wont have extensive access to engine

...
2. Script
...

cons[sic]:
would need an extensive database with commands

...
3. Compiled code
...

pros:
total access to engine


These are all different faces to the same problem. In all 3 cases, the DLL, Script, or compiled code will only have access to what you expose. Compiled code will not have "total access to engine" magically, you still need to expose function and data addresses in some fashion, just as you would with a DLL or scripted solution. The main difference between the 3 is that DLLs and scripts have developed proven solutions to accomplish such exposures.

How do you intend to expose your functions/data to your generated code? How do you intend to expose your generated code/data to your engine? How do you intend to accomodate the various calling conventions?

You seem to believe that generating code will magically take care of all the details and work just as easily as your compiler of choice, but that is not at all the truth. You'll have to do all the lifting yourself, make no doubt that what you would be in for is nothing less than creating a rather full-fledged compiler, linker and then some.

Share this post


Link to post
Share on other sites
wouldnt the compiled code have access to the system singleton?
if it would the whole engine would be open.

but it seems that pretty much everyone thinks that the compiled way is an unnecessary step, so ill fall back to using the dll implementation.
my idea was to create a usable IDE.
wich would give my engine an advantage vs other 2d engines.

well, ill create the graphical parts of the ide first (material creation, particle systems etc) and then ill do a little research on how hard / time consuming a compiler system would take to build.

Share this post


Link to post
Share on other sites
Just wanted to point this out:

The Unreal Engine's scripting language "Unreal Script" ist bytecode-compiled and according to Epic runs about 20 times slower than compiled C-code. Yet, a lot of the game logic in Unreal and UT is written in UnrealScript and both games are actually pretty fast.
So what's been said earlier: if a scripting language is too slow for your needs, you're probably trying to do something in scripting that you should be doing in-Engine.
Of course there are also scripting languages that do just-in-time compilation into machine code which gives you additional performance, but when done properly, the performance even of a (bytecode-) interpreted language should be sufficient for pretty much all gaming needs.

Share this post


Link to post
Share on other sites
Quote:
Original post by LackOfGrace
wouldnt the compiled code have access to the system singleton?
if it would the whole engine would be open.


No, not unless you expose it. Remember, executable code retains no symbolic information what-so-ever. Class, Variable and function names no longer exists, they are only addresses at this point. Class and struct definitions go out the window as well, only the class's constructor knows its size, and only the compiler knew how it was laid out in memory.

There is absolutely no way to hook into an executable and call "System::GetInstance()" in a straight-forward fashion. You need the address of System::GetInstance() and you need to know its calling convention. You need to know what parameters are expected and how they are arranged before jumping to the function (In this case, there are no parameters), next, you'll need to know what the function returns and how it returns it. Once you've gotten this far, you need to know how the returned object, presumably a pointer to a System object in this case, is laid out and where each of it's member functions are... oh, and don't forget to pass that hidden "this" parameter to all member functions.

This process must be repeated for every class, structure, function and variable the generated code needs access to. This is tedious and painstaking at best and is highly susceptible to mistakes.

The bottom line is that its not at all practical to impliment run-time code generation for scripting purposes. Most popular scripting solutions used in professional games today, such as Lua or UnrealScript, run on byte-code interpretters, usually implimented as either stack-based virtual machines or memory-to-memory virtual machines, even high-end games on limited devices (I know a recent FPS on the PSP uses Lua) use scripting, but they use it at an appropriately high level.

Share this post


Link to post
Share on other sites
Options 1 (DLL) and 3 ("compiled code") are the same. Unless you plan on magically reinventing the DLL mechanism, which is stupid (and more work than you're ready for). You can't take the object files produced by the compiler you "send" the code to and "embed" them into your currently running executable... or rather, you can (sortof) but the way you do that is via a DLL. Any other method would be very over engineered for the apparent scope of your game.

Scripting implementations, done well, are not as slow as you seem to believe. They can be quite efficient when used for the right things. It sounds like you're really suffering from a combination of (a) using them for the wrong things and (b) making assumptions about the performance of scripting in general.

Both are bad ideas. Use an existing scripting language, the performance will very likely be fine as long as you don't try to make the entire "engine" super-scriptable-zomg-you-can-change-anything (in other words, keep the scripting high level).

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!