Before starting with an MMO, have you looked at how such designs are made? The issues that need to be solved with regard to network transport, latency compensation, databases, persistence, horizontal scalability, hosting, backup and recovery, ....

With that in mind - why not just make server Erlang, and client C++. They'll be communicating via sockets anyway.

Quote:Original post by Konfusius
ejthayer makes a good point, aiming right at the core of the problem. An engine is often tailored to a specific genre, for various reasons. What kind of game is your engine's focus? Do you want to create an extremely generic engine, or a more specifc one?

Again, I stated that it wasn't an "engine" but more of a framework. It may be mostly semantics, but it's a slightly different mindset. It would be flexible enough that "genre" would have nothing to do with it. But if I had to give it a general focus, it would be on FPS and 3rd person games.

Quote:Original post by Antheus
Before starting with an MMO, have you looked at how such designs are made? The issues that need to be solved with regard to network transport, latency compensation, databases, persistence, horizontal scalability, hosting, backup and recovery, ....

With that in mind - why not just make server Erlang, and client C++. They'll be communicating via sockets anyway.

I have no idea where you got the idea this was for an MMO. An MMO isn't even remotely feasible as I lack the funding for 1000s of servers and terabytes of internet bandwidth.

Quote:
though I fail to see why it is necessary for you to know this.

Because regardless of what you want to call it, the goal of an infrastructural project like your engine/framework/whatever is to provide infrastructure. Furthermore, every project needs requirements. Somebody who has never built a game before will have approximately zero understanding of the kinds of requirements a game (of any particular genre) may require and are thus patently unqualified to be making infrastructural support for said requirements. While they may try, they will probably just waste their time. It is thus highly relevant to the discussion because the more games one has worked one, the better one will be able to extract a useful set of requirements.

So far, you have not provided any requirements -- you've provided some brief goals, but with no discussion behind them. Without further background there's not much else to say.

Quote:
It would be flexible enough that "genre" would have nothing to do with it. But if I had to give it a general focus, it would be on FPS and 3rd person games.

For example, how do you envision that working? There are some pretty serious differentials in techniques to overcome if your end goal is something that users interact with primarily via programming these 'object scripts.' You're basically saying that you'll be supporting several kinds of terrain engines, several kinds of geometry engines, almost every kind of animation implementation reasonable, et cetera -- because various genres of games are more suited to various implementations of all of those technologies (at least, if you want things to be reasonably optimal). It's tough to believe that somebody with a game development background consisting of "limited amount of OpenGL programming in a C++ / Unix programming environment" (beyond using other engines) would really be able to adequately understand that scope.

Therefore we must conclude you're either patently unqualified, and dismiss the thread as a humor, or that you just haven't clarified your goals enough. I'd rather go with the latter, personally, wouldn't you?

I will try to help you with your requirements by drilling into the goals you've specified:

Quote:
The core will be written in C++. I want to use erlang styled C# scripting for each game object or "actor."
...
One of the hardest things I see is how to interface the two; the C++ engine core and the C# object scripts.

Why write the "core" in C++ if the rest will use C#? Why not write it all in C#? Do you understand the cost of the interoperability layer enough to understand how to tame the most obvious alleged benefit of C++ ("speed?"); how much experience with C++/CLI do you have, or what other technology do you think you're going to use to leverage the interop? What benefit do you think a heterogeneous environment provides your end users that would outweigh those of a homogeneous environment?

Quote:
That is to say, each game entity will have it's own lightweight "thread" which executes cooperatively with the rest of the objects, in a linear manner.

What do you mean by "thread" and "linear manner?" Round-robin? What problem does this solve that's worth introducing the extra complexity?

Quote:
These scripts communicate with each other by passing messages. It would be event driven

Depending on your definitions of "event driven" and "messages" (and your planned implementation) these two statements could be the same thing, or at odds. Furthermore, how does the messaging play into the concurrency of the 'thread' updates and how do you avoid locking? If a "thread" is not actually related to a OS thread, what is it? Why?

Quote:
The idea is that the only programming the game developers should have to worry about is the object scripts,

See above; how is everything else handled then -- the terrain or levels? The animation? Content load? Hell, content production? Do you address these needs or not?

Quote:
The engine would be installed independently of the game.

Why?

Quote:
I was thinking one way to accomplish this may be to have be to have the game be a simple front-end written in C#. It's only purpose would be to host the object scripts and provide an execution point for the game.

Why?

Quote:
It would than communicate with the actual engine which is a separate process using IPC (perhapse named pipes).

Why?

Quote:
I knew this was the reaction I would get. I didn't come here asking what my project should be or if my project was worth it.

Irrelevant, as you don't own the thread.

Quote:
Why write the "core" in C++ if the rest will use C#?

If a game engine uses Python scripting, then why is the entire engine written in Python? I chose C++ for the engine because it's my main language of choice and offers the performance I want. I chose C# for the scripting because it is easier to use and offers the features I need. My goal is to abstract the core of the engine from the user and provide a easy to use interface that allows them to script a game with no knowledge of C++ or the underlying code of the engine.

Quote:What do you mean by "thread" and "linear manner?" Round-robin? What problem does this solve that's worth introducing the extra complexity?

If you read this article (http://harkal.sylphis3d.com/2005/08/10/multithreaded-game-scripting-with-stackless-python/), maybe it would make more sense to you. I take if you are unfamiliar with this concept. That, or I explained it poorly.

Quote:See above; how is everything else handled then -- the terrain or levels? The animation? Content load? Hell, content production? Do you address these needs or not?

It's handled by the framework, which included things like a resource manager (loads content for starters), AI, Physics, etc.

Quote:
Why?

I have 5 games installed on my computer that use Unreal 3, each with it's own local copy of the engine. Kinda redundant. You need to start thinking of this as a framework (like the .NET framework.) Maybe you think this concept is pointless and stupid, or maybe you're just old fashioned and resistant to new or different ideas. IDK. Maybe if I explained it better you would actually like it.

Quote:
If a game engine uses Python scripting, then why is the entire engine written in Python?

I don't know. That depends on the engine. What others have chosen to do should not really be relevant to what you choose to do unless there are, perhaps, compelling marketing reasons.

Quote:
I chose C++ for the engine because it's my main language of choice and offers the performance I want. I chose C# for the scripting because it is easier to use and offers the features I need.

The first point is fair. The second is, as well. But what about integrating them? You didn't address that issue when I raised it -- what is your plan for interoperability between the two, and do the available technologies offer the performance and features you require when combined?

Quote:
If you read this article (http://harkal.sylphis3d.com/2005/08/10/multithreaded-game-scripting-with-stackless-python/), maybe it would make more sense to you. I take if you are unfamiliar with this concept. That, or I explained it poorly.

You explained it poorly -- you described the issue in general terms that have multiple meanings depending on context, and you further obfuscated the meaning of those terms by putting quotes around some of them. I've done a considerable amount of work in concurrency, enough to be able to see several different meanings for your original statement. I'm trying to avoid making assumptions about your plans; you're not giving me much to work with.

Quote:
It's handled by the framework, which included things like a resource manager (loads content for starters), AI, Physics, etc.

How? So you really do have support for multiple terrain engines?

Quote:
I have 5 games installed on my computer that use Unreal 3, each with it's own local copy of the engine. Kinda redundant.

This goes back to the first point -- it doesn't matter what other people do. I don't care; I thought you wanted to talk about your engine, not theirs?

Quote:
You need to start thinking of this as a framework (like the .NET framework.)

No, you need to tell me (and the rest of us reading this thread) more about your plans so that we can get a better idea of what you're thinking. "Framework" is, like engine, an ambiguous term without context and you haven't provided context or background. You don't want me, or anybody else, to be assuming anything about your project if possible -- like for example with what happened with the MMO comments, above. That just results in pointless noise.

Quote:
Maybe you think this concept is pointless and stupid, or maybe you're just old fashioned and resistant to new or different ideas. IDK. Maybe if I explained it better you would actually like it.

I cannot tell what "this concept" is because you haven't described it very well. I get the "what" (to a degree) that you're trying to do, but I don't know why. You claim to "appreciate your thought on the matter," but you aren't responding to my queries for more in-depth information about why you are doing things -- or is the only reason you're picking this organizational structure because Unreal does something similar or because other engines use Python for scripting?

Quote:
what is your plan for interoperability between the two, and do the available technologies offer the performance and features you require when combined?

That I cannot answer. That was one of the main questions I had when I came here and is one of the largest hurdles I have to overcome before I can even start to go anywhere with this.

Quote:
You explained it poorly

Not surprising. It's still a new concept to me. I'd by lying if I said I fully grasp it at this point.

Quote:
How? So you really do have support for multiple terrain engines?

What do you mean by support for multiple terrain engines? I was thinking that terrain could either be an object who's geometry is loaded as BSP or mesh data, or generated automatically by the engine using height maps and basic properties. In any case, for flexibilities sake, there will be more than one option.

Quote:
This goes back to the first point -- it doesn't matter what other people do. I don't care; I thought you wanted to talk about your engine, not theirs?

As far as I know I'm not basing this idea of any current engine. I can't help but bring up the occasional example from the real world though to illustrate a simple point.

The basic idea in a nutshell is to create a framework that allows someone to create a game without having to touch a single C/C++ compiler. The only programming they should have to do is provide some simple scripts that define how the game object act. Simple enough that a non-programmer should be able to write them. This would be done through an IDE interface. They start up a new project, add all their objects, write some basic scripts to define what those objects will do using a simple script interface, and add resources like textures and meshes to those objects. They can configure the objects properties, like whether or not the object will be controlled by the physics engine or AI. If so, they can choose from plethora of options available to an object that uses physics or AI. I really don't know how to explain this any better if you aren't getting the general idea. The closest thing I know of to this currently in existence is the source engine (oh no, real world example, don't hate) although even this doesn't take the idea quite as far as I'd like.

Quote:
I knew this was the reaction I would get. I didn't come here asking what my project should be or if my project was worth it.

I understand your frustration here. I have posted things only to have people tell me not to do it that way. Really I do not want their opinion, I just wanted some help.

So let me kinda rephrase things.

Instead of setting out to write just an engine try writing your engine with something to apply it to. Writing your engine with a FPS will help you refine the engine more and program for pitfalls and problems you might not have otherwise seen. In the end you have your engine and a small shooter to possibly start a game with. Then maybe take the engine and apply it to creating an RPG type of world where you should be able to refine it even more.

Either way I think you will find that some parts of the engine will require a bit of molding to each idea you want to use it for.

Quote:Original post by ejthayer
Instead of setting out to write just an engine try writing your engine with something to apply it to. Writing your engine with a FPS will help you refine the engine more and program for pitfalls and problems you might not have otherwise seen. In the end you have your engine and a small shooter to possibly start a game with. Then maybe take the engine and apply it to creating an RPG type of world where you should be able to refine it even more.

Either way I think you will find that some parts of the engine will require a bit of molding to each idea you want to use it for.

I did have a FPS game in mind that would be created alongside this engine (but it's not the reason I want to create the engine, so I'd prefer people not tell me to simply create the FPS instead) that would be a bit of a show real for the engine.

Quote:game without having to touch a single C/C++ compiler. The only programming they should have to do is provide some simple scripts that define how the game object act. Simple enough that a non-programmer should be able to write them.

That made me chuckle.

Making it easy enough for "non programmers" was the purpose of the first programming language 53 years ago. I've heard it all of my short life. I remember when I was a kid hearing about this new Java thing that would let people make software without programming.

It doesn't work that way, though.

If your engine or kit or framework or whatever is so abstract and void that it supports any design, than it in and of itself has no behavior.

You must express the behavior rules to the computer in a manner the computer can understand. You have to tell the computer what to do. You have to program.

[Edited by - JoeCooper on April 7, 2010 4:31:29 PM]

Quote:
That I cannot answer. That was one of the main questions I had when I came here and is one of the largest hurdles I have to overcome before I can even start to go anywhere with this.

Issue the first with using C# as a scripting language, regardless of the interoperability solution, is that the C# assembly must be loaded into an appdomain and can never be unloaded without tearing down the entire appdomain. This means hot-reload of your object scripts is very heavy-weight, because all scripts must be unloaded to unload any script, unless you put each script in its own appdomain, which is probably worse because appdomains are by design and isolation layer, and communicating between them is harder than communicating within them.

Interoperability between native (C++) and managed (C#) code is generally done in one of three ways: Platform invoke, COM, or C++/CLI. Platform invoke is very limiting and would require you to package -- either at build time or run time, all the C# into a compiled DLL or set of DLLs. It's probably non-workable for scripting.

COM is disgusting and there are generally better options, but it's worth considering since it would not require you to use C++/CLI, which will itself mean you use C++/CLI completely (no pure C++, although you can still write pure native code) or you have a C++/CLI DLL you load to execute scripts in your framework code.

C++/CLI is the best option. However, it is a horrid, complex language (worse than C++, even). And there are performance considerations to worry about -- C++/CLI is potentially the fastest way to make the native/managed transitions you'll require to implement this, but it actually achieving that optimal performance requires a non-superficial understanding the boundary between managed and native code, and how to bridge it properly.

All of the above assumes you'll want the scripting running in-process. You could alternatively build a processor process that is fully-managed and executes the C# scripts, obviating the need for direct interop. However you'd then have an IPC communication problem; you'd have to serialize the game state across into the shim process and serialize it back every so often, or propagate updates via incremental notifications, much as if you were writing a client/server system.

All in all C# is probably a poor choice for a scripting language, because these solutions are pretty heavy. It's certainly doable, but... is that really what you want to be spending your time on this project on? Are there any other languages that meet your scripting requirements besides C#?

Quote:
Not surprising. It's still a new concept to me. I'd by lying if I said I fully grasp it at this point.

The Stackless tasklet model is interesting and certainly has use. However you do need to consider that most Python implementations, including Stackless, have a GIL (global interpreter lock) that ensures only one core is executing bytecode at a time, which means you don't get the benefit of actual multicore processors until you use multiple interpreter processes. Presumably you don't want to adhere that strictly to the model, so you need to have a good understanding of concurrent programming techniques. In particular you're going to want to have a solution for handling object updates that doesn't require objects to take locks on global state very often.

Quote:
What do you mean by support for multiple terrain engines? I was thinking that terrain could either be an object who's geometry is loaded as BSP or mesh data, or generated automatically by the engine using height maps and basic properties. In any case, for flexibilities sake, there will be more than one option.

A third-person shooter, a flight sim, and an RTS have very different needs from terrain. To support all these games with your desired model of writing only object scripts, the user basically needs to be able to either:

1) check a box that says which type of terrain, of the N available, to use. Great for the user, since they get to pick the type optimal for their own scenario. Sucks for you, since you have to implement N different types of terrain engines.

2) check a box that says "use terrain," and they get your one implementation. Great for you, you only write it once. Sucks for them, they get the least-common-denominator.

Your solution of making the terrain just another object may or may not work, depending on how extensive you actually want these 'object scripts' to be. Optimal terrain generally has very different update/rendering/partitioning/collision needs than the other objects in the game world, and you may end up with a very simplified / nonperformant / nonscalable solution, which may render your framework useless to a large variety of users. But you haven't spoken much about the interface(s) available to these object scripts, so this may be a nonissue.

Quote:
As far as I know I'm not basing this idea of any current engine. I can't help but bring up the occasional example from the real world though to illustrate a simple point.

The closest thing I know of to this currently in existence is the source engine (oh no, real world example, don't hate)

Providing an example is quite different from providing a requirement. Saying "my project will be like the Source engine," is not the same as saying "my project will put the engine code in an executable and the game code in a DLL because the Source engine does it(*)." The latter is a statement about a requirement, and it's a stupid one. It's a requirement based on the "me too, I'm cool too, really, see how much like the Source engine I am!" concept, instead of actual, rationale arguments about how the requirement will add value for users or developers.


(*) I don't actually know if the Source engine does this, but the actual engine isn't the point.


I ask these questions because there are reasons that other developers, for example, the Unreal guys, made these decisions. They're not arbitrary. But "because Unreal did" is, if not arbitrary at least showing a lack of thought about the decision. The model whereby the engine is the .exe and the game is the .dll increases complexity and thus increases development cost, and the advantages to you are...? What, exactly? And have you thought through the ramifications?

Consider your Unreal counterargument again. Five copies, huh? What makes you think you won't need five copies of your engine executable? What if you have five different versions -- because your product is a success. Breaking changes and incompatibilities between versions are a reality of software development, so clients of your code will probably be installing specific versions of your framework anyhow (it's what .NET does, for example). If avoiding duplicate copies of your framework on client machines is your real requirement, putting the framework in a DLL works just as well or better and requires less effort on both your part and on the part of the client.

Also, have you seen Unity? Unity can do something very similar to what you're talking about (among other things)