Thoughts on this component system for a game engine.

This topic is 3187 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

Recommended Posts

I have been working on my new game engine that will be based around the component based entity system. I would like to know what everyone thinks of this system for implementing the components. The Components only store data and the engine Systems contain functionality. The Engine class contains a static Component Manager class that acts as a database with the components being stored in a vector. The Component Manager has functions like GetComponentsByEntity that takes a input of a string entity name and loops through all of the components, fills a output vector with any Components who's EntityName variable is the same as the input, and returns the output vector. There are three other functions: GetComponentsByType, GetComponentByName, and GetComponentByTypeAndEntity. As an example of usage the Render System has an octree that is created at start. Every time the Scene changes(Scene is just a part of the engine state manager of game states), the Render System get's all of the objects that need to be rendered by calling GetComponentsByType("Renderable"). Then every frame(or possibly only when receiving an update message) the Render System determines where the Renderables are located in the octree by iterating over all of the Renderables and calling GetComponentByTypeAndEntity("Positionable", renderables->GetEntity()) and using the output to place the Renderables vector id into the BranchNode of the octree. There will be a fixed set of Component Types that will be able to be used. Game logic will be done in Script Components that load up a lua script. I would appreciate and comments, suggestions, and constructive criticism that anyone has to offer. Thanks. :)

Share on other sites
sounds hideously slow and complicated
string compares? in your render loop? come on man!

if you want to have as single list of items, and have those items mean different things, perhaps you should look at a spanning array. although you have to iterate through a list of objects to update them, you may as well check if you can render them at that point in time.

sounds like your trying to go the DOM sort of direction, where you have a cloud of stuff and filter it based on what you want to do. sure it sounds like a good idea, but really, it tends to be a hideous mess of untyped crap. and ive seen engines try it (even commercial ones).

keep it simple.. keep it explicit.. dont try to hide complexity behind even more complexity.

Share on other sites
I had a feeling that this implementation was not the greatest way to manage this. :( I am still not the most experienced of coders and sortof just wanted to start planning how i wanted the engine to be used and then ask more experienced coders for advice on how to optimize it. :P

Thanks for the advice about spanning arrays. I will look into that. But everyone please keep the ideas coming. :)

Thanks

Share on other sites
Quote:
 Original post by Matt_Dsounds hideously slow and complicatedstring compares? in your render loop? come on man!

The above concept can be implement as 'return renderables;', so this "hideously slow" is a single pointer assignment.

In general, the problem of GetComponentByType can be implement in a completely trivial manner - each component registers with its relevant system. So Renderable puts itself into Renderable list. And so on.

Even more, components can be allocated inside this list, which means that they can be traversed very efficiently.

The biggest bottleneck comes from ad-hoc queries and arbitrary lookups, where you are constrained to whatever indexing scheme is used. These will be some function of n, and frequently impacted by cache misses. Fortunately, these will be limited to logic, which is rarely a bottleneck, even if implemented completely in scripting language.

Share on other sites
I have been thinking about other possible solutions and i have a couple questions. I am now starting to think about possibly just having a Map of the for each type of component inside of the Systems rather than a single vector of components.

For an example i will go back to the Render System. The Render System has a member variable of a std::map<std::string entityName, Renderable component>;. Whenever the user of the engine calls Engine::AddRenderable(renderableDesc);, the Engine just has the Render System add a new Renderable with the members passed by the renderableDesc. The same thing happens with the Movement System and the Movable components. Then whenever the Render System needs to get the position it just requests it straight from the Movement System with a call to Engine::GetMovable(std::string entity);, which just fetches the Movable from the MovementSystem Map by using the entity key.

Here are my questions: #1. Does this sound like a good plan? #2. It seems as if the best way to implement this would be with static functions for the Add???() and Get???() functions. Would there be any problem with doing that for a dll Game Engine?

Thanks again for the help.

Share on other sites
I use the component system and love it. It helps keep all the logic separate, so a game object will have a base entity component, a basebrain component and a Base Physics component (physics, graphics, AI, etc). If you make an abstract interface you can easily change the entire portions of the engine without any trouble at all (IE I can easily change from Havok to physX or whatever without much bother).

Share on other sites
Ahh, "component system." Sounds awfully complicated.

struct PositionComponent {    float x, y, z;};struct RenderComponent {    PositionComponent &pos;  // a required subcomponent};struct Entity {    PositionComponent pos; // a member subcomponent    RenderComponent *r; // an optional subcomponent};

Then you could maintain a list of RenderComponents to draw each frame, or a list of Entities to go through each frame and do things for each component.

Dunno why everyone always goes and makes this all complicated with ids and names and stuff. What's wrong with pointers/references/members at this level? EDIT: Unless you're trying to do this across threads.

Share on other sites
What if not all entities have RenderComponents? You've wasted a pointer. What if you have lots and lots of component types? You've wasted lots and lots of pointers. What if you want an entity to have multiple components of the same type? Make it an array? What if you want to add and remove these dynamically? Tough bits.

It's easy to say "look, it can be this simple!" if your problem is also very simple, but games aren't that simple.

Share on other sites
Quote:
 If you make an abstract interface you can easily change the entire portions of the engine without any trouble at all (IE I can easily change from Havok to physX or whatever without much bother).

I plan for my engine to be able to do this by having the Systems be separate from the wrapper. As an example with my plan for physics: the Physics System isn't a wrapper around anything, instead it is just what runs the simulation. Slightly separated from the Physics System are the interfaces like PhysicsScene and PhysicsJoint. Those interfaces are the wrappers around the api functionality and the Physics System just manages them. Kinda hard to explain.

Share on other sites
Quote:
 Original post by hymermanWhat if not all entities have RenderComponents? You've wasted a pointer. What if you have lots and lots of component types? You've wasted lots and lots of pointers. What if you want an entity to have multiple components of the same type? Make it an array? What if you want to add and remove these dynamically? Tough bits.It's easy to say "look, it can be this simple!" if your problem is also very simple, but games aren't that simple.

Dare I say: "Who cares?"

I'll waste a million pointers if it means that I don't have to spend an extra hour poking around through a tangled mess of
((ThingieComponent)entity->;GetComponent[COMPONENT_ID_THINGIE])->;DoSomethingToAnotherEntity(somethingElse);...class ThingyComponent : Component {    void DosomethingToAnotherEntity(Entity& entity) {        ((PositionComponent)entity.GetComponent[COMPONENT_ID_POSITION]).Translate(1, 2, 3);    }}

Games are complicated. Why further complicate them with complexities that aren't necessary. Go ahead and waste a meg (or shall I say 'mib?') of RAM on unneeded pointers. Maybe you'll force the user to upgrade? [We don't need off-handed political and/or racially coloured comments in here, thanks - Zahlman]

[Sorry! Wasn't thinking. Patrick]

[Edited by - smr on June 2, 2009 9:37:20 PM]

Share on other sites
Or instead of doing messy code like that you could make an easy to use component system and just call:

PositionSystem.Translate("EntityName", 1, 2, 3);

or

PositionSystem.Translate(PositionID, 1, 2, 3);

Simple right? And probably in an ideal component system, all you would need to do is add some components and call functions like these from within your code or scripts.

Share on other sites
Quote:
 Original post by hymermanWhat if not all entities have RenderComponents? You've wasted a pointer. What if you have lots and lots of component types? You've wasted lots and lots of pointers. What if you want an entity to have multiple components of the same type? Make it an array? What if you want to add and remove these dynamically? Tough bits.

You should design your systems so that each component doesn't depend on a huge number of other components. If your components has more than a few optional subcomponents, I'd argue that it's time to refactor.

Yes, if you're going to be restructuring your component hierarchy at run-time, you'll need that extra layer of abstraction. However many people think "component system" and immediately go toward a Dungeon Siege or Game Programming Gems solution, when classes support composition quite well.

Share on other sites
I'm using a component based entity system as well but a liitle bit different.

Here a entity is only a ID for the programmer and internally its a component container.
Components can when they belong to a entity, register for update and input calls.
Components talk to each other across a event system driven by the entities.

CompEntity entity;
entity->FireEvent("changePosition", senderEntity(can be NULL), DATA);

DATA can be of any type i use something similiar to boost::any but its not boost::any.

Components are attached and deattached via the ComponentManager.

You can access Entities by id. Every Entity has a unique id so no look up tables just simply array calls. Same goes for ComponentQuerries. Components have TypeIDs and names to create them on the fly.

Share on other sites
I scanned through your source code Wizecoder, and I have a couple questions/thoughts:

1) How are you planning on integrating a scene graph? This has been something I have struggled with to the point that I'm thinking of forgetting hierarchical culling based on relations (which is typical of a scene graph) and just being happy with the spatial culling of an octree like you apparently going for.

2) Overall I really like your design, and it goes along with how I pictured things. There is one issue that I am wondering if there isn't a better solution to and that is, for example, how your render system calls BMGPositionable::GetPosition(). This introduces a dependency of the render system on the actual data lying in a component that could possibly be used by another system, like physics. When you start multi-threading, you will assumably be assigning a thread to each system. Now you will have a problem since both the render system and physics system will be operating on the same shared data.

The only solution I see to 2) is to have redundant data stored in components. For example, both the render and physics components would need to have position data stored in them. Then, when the actual position component changes, it could broadcast a "PositionChanged" message that will modify the position data in both the physics and render components. The question is, does the separation between systems that this will achieve justify the data redundancy?

Share on other sites
#1. I was thinking that once the scene graph starts being implemented then there could just be a new component type called HeirarchyComponent or something like that. Then a HeirarchyComponent could just be added to the Scene Graph and the RenderComponent could be added to the Octree.

#2. I haven't thought to heavily about multi-threading yet but you could probably just protect some of the data by locking it. This will be something i will probably think more about a while from now. And about the message system you mentioned, i am thinking i might do something like that. When one of the functions like PositionSystem->Move("EntityName", 1, 2, 3); is called the Position System would send a message to the necessary systems which will then update there copy of the data(like the Graphics System changing the node locations in the Octree).

Share on other sites
Components on your entity sounds fine, just maintain a hash_map or vector on the entity so you can loop through and update your components. Also, I'd try and make a nice interface to communicate between components. Having to access the vector directly and find the component you want, to make a function call would be tough. A messaging system to be able to send a message and let the component that needs to handle it might be the way to go. Especially if you want one message to be handled by multiple components.

Share on other sites
several problems i c here.

1. the system has a incrediblely low cohesion. everything is tightly coupled together.

2. this issue is somewhat introduced coz of the low cohesion. ur system will have a central bottle neck at the Engine class or Manager class. this applies to all manager oriented systems, which ultimately will run into scalability problems.

3. if u r trying to make a high quality engine, i would suggest u redesign and probably start over by studying some of the open source or cheap commercial engines. it seems like there isnt really a design here yet.

4. multithreaded architecture. well since the design is very incomplete, i c it as being almost impossible to scale up to a multithreaded level. without multithreading on current hardware (pc/mac or consoles), the engine will not perform.

its a good start, but theres a LOT to be done before u can code anything. try to study some popular engines before u dive into engine development. make a medium sized game using a popular engine would be a good way to learn.

good luck

Share on other sites
Quote:
 Original post by neakorseveral problems i c here.1. the system has a incrediblely low cohesion. everything is tightly coupled together.good luck

This doesn't make sense. Cohesion is proportional to coupling. What Wizecoder has going right now is a great start. I would disagree with your assessment that he needs to start over from scratch.

Quote:
 Original post by WizecoderI haven't thought to heavily about multi-threading yet but you could probably just protect some of the data by locking it.good luck

I agree, a little bit of locking would probably do the trick and is likely the best solution. I do however really like the idea of %100 of the communication via message-passing for networking ease and multithreading. Shared locking just seems to go against my "ideal" view of how a component design should be. Yes I know this may not be realistic, but one can dream can't he? In fact... I think I am going to give it a go tomorrow and see what a strictly message-passing engine would look like.

Share on other sites
Quote:
Original post by Matt Sloan
Quote:
 Original post by WizecoderI haven't thought to heavily about multi-threading yet but you could probably just protect some of the data by locking it.good luck
I agree, a little bit of locking would probably do the trick and is likely the best solution. I do however really like the idea of %100 of the communication via message-passing for networking ease and multithreading. Shared locking just seems to go against my "ideal" view of how a component design should be. Yes I know this may not be realistic, but one can dream can't he? In fact... I think I am going to give it a go tomorrow and see what a strictly message-passing engine would look like.
Done naively, that 'little bit of locking' just axed any performance gain multi-threading on this level would have gained.

An implicit message passing system works pretty well - I find explicit messages to be too expensive for inner loop calculations. By implicit I mean that each piece of data is visible to all sub-systems, but many only be written to by exactly one (predefined) sub-system.

If you can enforce that one constraint in your design, you can freely spin off sub-systems into their own threads, with only minimal locking/buffering required to maintain consistency.

Share on other sites
Quote:
 Original post by swiftcoderAn implicit message passing system works pretty well - I find explicit messages to be too expensive for inner loop calculations. By implicit I mean that each piece of data is visible to all sub-systems, but many only be written to by exactly one (predefined) sub-system.

This is the other option I was thinking about, and seems to be one of the more popular routes.

Few questions..

1) Tell me if I have your implicit system understood based on this example. The position component contains data that needs to be read by both render and physics components. When physics wants to change position data, it sends out a message that is handled by the position system. The position system in turn locks the data, makes changes, then releases the lock. If the renderer wants to read that position data it would do so via a const Vec3& getPos() function which would be accessed through entity properties so you end up with no coupling between render and position systems.
How does that sound?

2) How inefficient did you find explicit message passing to be exactly? Is there anyway it could be viable even if it was slower than implicit?

Share on other sites

IEntity contains a map of IComponent instances. IComponent could be GameLogic, Physics, or Graphics (stores texture file, mesh file, the IRenderable instance) and other components needed. Each IComponent implementation has it's own manager, like GameLogicManager, PhysicsManager, RenderingManager.

There are two types of managers, the game update managers and rendering manager. The rendering manager only needs to know the position, the IRenderable instance (handles the rendering of each entity depending on its implementation whether mesh or texture), and the bounding information (AABB or sphere or whatever). The GraphicsComponent is the suited component to have these data. It's up to the rendering manager whether to use octree culling, view frustum culling, and so on.

So the game loop will look like:

void update() {  gameLogicManager.update(); // performs logic on GameLogic components  physicsManager.update(); // performs logic on Physics components  // other update managers calls update}void render(){  renderingManager.render(); // it's up to the renderingManager how to render stuff}

I also have a concept of IEntityTemplate objects, like ShipEntityTemplate. This class loads the required components a certain entity needs.

Share on other sites
Quote:
Original post by Matt Sloan
Quote:
 Original post by swiftcoderAn implicit message passing system works pretty well - I find explicit messages to be too expensive for inner loop calculations. By implicit I mean that each piece of data is visible to all sub-systems, but many only be written to by exactly one (predefined) sub-system.
This is the other option I was thinking about, and seems to be one of the more popular routes.

Few questions..

1) Tell me if I have your implicit system understood based on this example. The position component contains data that needs to be read by both render and physics components. When physics wants to change position data, it sends out a message that is handled by the position system. The position system in turn locks the data, makes changes, then releases the lock. If the renderer wants to read that position data it would do so via a const Vec3& getPos() function which would be accessed through entity properties so you end up with no coupling between render and position systems.
How does that sound?
I meant a whole lot more implicit. In particular, I have never seen the point of trying to de-couple the position component, because every entity has one, and most other components need access to it.

So there is no position component, just a position attribute (which resembles a component in terms of API) attached directly to the entity. The physics component writes directly to it (using minimal locking), and all other components read directly from it.
Quote:
 2) How inefficient did you find explicit message passing to be exactly? Is there anyway it could be viable even if it was slower than implicit?
It depends how you arrange it. Your system requires a message to be passed every frame, per-shared-attribute, per-entity, and that is one hell of a lot of messages. Still, with intelligent caching of data, you can probably get away with it.

Share on other sites
I know that Singletons are not recommended but i am starting to think that the best option for the main Engine class would be for it to be a Singleton. Thoughts on this?

Quote:
 So there is no position component, just a position attribute (which resembles a component in terms of API) attached directly to the entity. The physics component writes directly to it (using minimal locking), and all other components read directly from it.

Actually how i am planning it this might not work because i don't want there to be any physical Entity class. An Entity is just a label that is a member of the Components.

Thanks for all the help so far.

Share on other sites
Quote:
 Original post by WizecoderI know that Singletons are not recommended but i am starting to think that the best option for the main Engine class would be for it to be a Singleton. Thoughts on this?
Why would it need to be a singleton? I very much doubt that every class in your code is going to need to access the Engine class (and that is the only truly justifiable reason to use a singleton/monostate).
Quote:
Quote:
 So there is no position component, just a position attribute (which resembles a component in terms of API) attached directly to the entity. The physics component writes directly to it (using minimal locking), and all other components read directly from it.
Actually how i am planning it this might not work because i don't want there to be any physical Entity class. An Entity is just a label that is a member of the Components.
Ja, I took that approach initially as well, but I ended up reverting to storing a few pieces of very fundamental data in the entity.

Share on other sites
Quote:
original post by swiftcoder:
Quote:
 Original post by WizecoderI know that Singletons are not recommended but i am starting to think that the best option for the main Engine class would be for it to be a Singleton. Thoughts on this?

Why would it need to be a singleton? I very much doubt that every class in your code is going to need to access the Engine class (and that is the only truly justifiable reason to use a singleton/monostate).

It might be necessary for the Systems to be able to access some of the functionality within the Engine. I guess i could just store a pointer to the Engine inside the Systems but i was thinking a Singleton would be better.

Quote:
Quote:
Quote:
 So there is no position component, just a position attribute (which resembles a component in terms of API) attached directly to the entity. The physics component writes directly to it (using minimal locking), and all other components read directly from it.

Actually how i am planning it this might not work because i don't want there to be any physical Entity class. An Entity is just a label that is a member of the Components.

Ja, I took that approach initially as well, but I ended up reverting to storing a few pieces of very fundamental data in the entity.

I think i will still stick to as separate as possible. I can try to enforce creation of a position component if i make any level editor tools.