Critique my component-based entity system

Started by
14 comments, last by theOcelot 14 years, 10 months ago
After reading the Cowboy Programming article concerning component-based entity systems, I decided to implement my own for a small 2D game. This kind of design is probably overkill for such a simple project but I think it is a fun exercise and will help me enforce modular software design. In addition to the usual references on the subject I have also read Sneftel's thread, weighting the pros and cons of each method presented. I came up with the following requirements for my architecture:
  1. Type-safety must be preserved (no dynamic_cast<> or worse, static_cast<>).
  2. Components should be as independent as possible.
  3. Component and entity creation should be doable dynamically at run time in order to facilitate the integration of a contents editor.
  4. Communication between components of a same entity should not be done using packed (serialized) events and giant switch cases to determine their type (similar to point 1).
  5. Entities should be nothing more than a name common to several components (data and behaviour should be factored in components).
On aspect I liked in several implementations of this system type was the presence of subsystems (call them what you will) that manage particular kinds of components. For instance, the RenderSubsystem will manage RenderComponents, PhysicsSubsystem will manage PhysicsComponents and so forth with other component types. Since the subsystem contains a list of the components it manages, communication between same-type components of different entities is fast and efficient. In my implementation, components are indexed in an associative array using the names of their respective entities. For example, the PhysicsSubsystem has direct access to all PhysicsComponents, allowing it to check for collisions between them without having to fetch the components from their owner entities. Subsystems are also responsible for component creation. In my test implementation, the parameters passed to the component's constructor are hard-coded, but a future improvement will pass an XMLNode*, or something similar, containing the relevant information needed to create a component. All subsystems derive from a Subsystem abstract base class which defines two methods, Update() and Create(). Components, however, don't have a base class. Their managing subsystems are aware of their interface and don't have to use dynamic_cast<> all the time. The most tricky part of any component-base design is, in my opinion, the communication between the components forming a single entity. For example, a ScriptComponent could ask the PhysicsComponent to move in a certain direction, which in turn should tell the RenderComponent that it moved to a certain position so that it can be drawn at the correct location on the screen. Most implementations store the positional information in the entities themselves since several components need that information, but one of my requirements was that an entity should not be anything more than a name or an ID. The main question now is how to implement a communication system while preserving type-safety (1, 4) and component independence (2). The observer design pattern (or another signal/slot system) is an obvious answer. The problem of "connecting" components to each other while keeping components independent, however, remains. My solution lies in the use of the mediator design pattern, which acts as a communication hub for the whole system. The mediator contains lists of listeners for each event type (e.g., Move or Hit) and the methods used to notify them. In order to preserve type-safety, I decided to go Java-style and use *EventHandler interfaces with abstract Handle*Event() methods. At creation time, components (implementing all relevant *EventHandler interfaces) subscribe to the events they can handle through the mediator. There is one Mediator instance for each entity, so that whenever a component receives an event notification it knows it comes from a component of the same entity. To take care of the 3rd requirement, I implemented an EntityBuilder class, which contains a map of Subsystem pointers indexed by their type. This class has 3 methods: RegisterSubsystem(), BuildEntity() and AddComponent(). RegisterSubsystem() is used to associate a concrete Subsystem subclass to a type string (e.g., subsystems["render"] contains a pointer to a RenderSubsystem). BuildEntity() stores an entity name and creates a new Mediator. AddComponent() is then used to call the Create() method on the correct subsystem (the component type is passed to AddComponent()). [UML diagram] In conclusion, here is a summary of my approach.
  • Type-safety is preserved throughout the entire system. There is no casting nor RTTI used anywhere in the system.
  • Components are completely independent. They subscribe to a Mediator for events they can handle and wait for them to occur. If no component can fire those specific events, the system still works without any problem.
  • Entities (components) can easily be created at run time. The integration of a content editor will be a trivial task.
  • Entities can only have one component of each type. This could easily be solved by using a mulimap instead of a map to store components in each subsystem.
  • The mediator object can quickly become a monolithic object. This is the main problem I have with this design, but I think it's a fair compromise for the advantages it offers (type-safety, component independence and dynamic entity creation).
I would appreciate your opinion and comments regarding my system and, if I may be so deserving, suggestions as to how it could be improved. I can post the code if anybody is interested.
it seems rational thought has been killed and buried
Advertisement
Quote:Original post by XezekielX
The most tricky part of any component-base design is, in my opinion, the communication between the components forming a single entity. For example, a ScriptComponent could ask the PhysicsComponent to move in a certain direction, which in turn should tell the RenderComponent that it moved to a certain position so that it can be drawn at the correct location on the screen.

The main question now is how to implement a communication system while preserving type-safety (1, 4) and component independence (2).
A simple answer is for the RenderComponent to inherit from an interface (e.g. IMovable) and for PhysicsComponent to have a member variable:
IMovable* m_RenderObject;
It can then communicate with another component pointed to by this variable with no overhead (once plugged in).


Quote:The problem of "connecting" components to each other while keeping components independent, however, remains.

My solution lies in the use of the mediator design pattern, which acts as a communication hub for the whole system. The mediator contains lists of listeners for each event type (e.g., Move or Hit) and the methods used to notify them.

At creation time, components subscribe to the events they can handle through the mediator. There is one Mediator instance for each entity, so that whenever a component receives an event notification it knows it comes from a component of the same entity.
The system that I use has a simmilar approach.
Each entity contains a registration list of all of the interfaces provided by it's components. When created, each component registers it's interfaces with the entity.
Any component can then query the entity for a particular kind of interface.
void PhysicsComponent::Update( Entity& parent ){  if( !m_RenderObject )    m_RenderObject = parent.GetInterface<IMovable>();  if( m_RenderObject )    m_RenderObject->Move( ... );}


Quote:

  • Entities can only have one component of each type. This could easily be solved by using a mulimap instead of a map to store components in each subsystem.

I've used systems in the past with this limitation, and it can get quite annoying in the long run (breaking entities into two entities for no reason...) I would recommend finding a work-around if possible.
For example, with my interface system they can be referenced by index.
size_t size = parent.CountInterface<IMovable>();for( size_t i=0; i<size; ++i ){  parent.GetInterface<IMovable>(i)->Move( ... );}


Quote:Original post by Hodgman
Each entity contains a registration list of all of the interfaces provided by it's components. When created, each component registers it's interfaces with the entity.
Any component can then query the entity for a particular kind of interface.
void PhysicsComponent::Update( Entity& parent ){  if( !m_RenderObject )    m_RenderObject = parent.GetInterface<IMovable>();  if( m_RenderObject )    m_RenderObject->Move( ... );}





Sorry to hijack, but some questions.

Hodgman, I am wondering what your interface registration system is like? We currently have a large, working component entity system without interfaces - people gain access to specific components, or use a broadcast-style messaging system to send messages to all of their siblings. I would like to add interface support, since it removes the dependency on component types and broadcast messages in favour of functional interfaces. I imagine that your templatized method Entity::GetInterface<T> must use T and extract a type-id to look it up - is it simply up to the child components to call Entity::RegisterInterface<T> so the same type-id can be used? Do you store it in a binary tree? I had thought to do a search across all the components like so:

T* Entity::FindInterface<T>(){   foreach component   {       return reinterpret_cast<T*>component.FindInterface( TasInterfaceID<T>() )   }   return NULL;}void* TransformComponent::FindInterface(int interfaceID){   if ( interfaceID == TAsInterfaceID<ITransform>() )   {      // cast to get correct multiple-inheritance base adddress      return static_cast<ITransform*>this;   }   return NULL;}


This has the benefit of not requiring storage in the entity, but a linear traversal would require clients to hold onto the interface for performance reasons.

How much memory do you give up to your registration? Is it by type - each type of entity has only one registration table? We do this right now for our component list - each type of entity has a table of offsets shared with all instances of that type of entity.

Would be curious to know.

As far as your original points, I think you are on your way, but I think sometimes people over-think this stuff and sweat the details. Everything seems on track except this one:

Quote:'Entities should be nothing more than a name common to several components (data and behaviour should be factored in components'.


I had this requirement in the past and boy did it not work out in the long run. We had cases where engineers were writing PropStateComponent1, PropStateComponent2, PropStateComponent3 just to get around it. Most of our components register themselves during their constructor with the various subsystems that care about them, so having multiple instances of the same type is not usually a problem. Now PropStateComponent looks for a sibling component of type 'PropStateController' and calls PropStateController::RegisterState( this ). Effectively each state just tells the controller about itself, whether there are one or twenty states. I can't imagine going back.

The other big thing that really simplifies things is to have a component that just stores data for the transform - trying to interface physics to rendering to animation without a common data repository is unnecessary hardship - we also do this with our animation pose hierarchy so different components can gain access to the animation join matrix hierarchy regardless of whether they read the data (rendering), create the data (animation), modify the data (headtracking), rely on the data (prop-placement).

Finally, one thing that we use a lot to solve some type safety issues. We have an in-house scripting system which is type safe. Our entities are connected together using a signal-slot mechanism that lets you target a named parameter-less signal from one component to a named parameter slot. This is all setup in our level editor. The connections are bound with our scripting language where we can guarantee at pipeline time that the arguments to the connection are type safe, including the parameters. We then compile down our script and it becomes a function that takes two void*'s which are bound to the source and target through strong name-checking. Since everything is built and compiled in the pipeline, we have guaranteed type safety even though there are a lot of casts at runtime: we get great performance and never have to worry about type issues. I point this out because we actually use our scripting system for defining object entities and gluing the types together in a type-safe way - the scripting language lets us write our own syntax for defining components as well as accessing things. The performance hit is really low since its only used to instantiate the object and populate C++ objects, but it does let us write some components in script.


Best of luck with your project!

-S.
Thanks to both of you for your input.
Quote:Original post by Hodgman
A simple answer is for the RenderComponent to inherit from an interface (e.g. IMovable) and for PhysicsComponent to have a member variable:
IMovable* m_RenderObject;
It can then communicate with another component pointed to by this variable with no overhead (once plugged in).

The system that I use has a simmilar approach.
Each entity contains a registration list of all of the interfaces provided by it's components. When created, each component registers it's interfaces with the entity.
Any component can then query the entity for a particular kind of interface.
void PhysicsComponent::Update( Entity& parent ){  if( !m_RenderObject )    m_RenderObject = parent.GetInterface<IMovable>();  if( m_RenderObject )    m_RenderObject->Move( ... );}
What if a PhysicsComponent doesn't need a RenderComponent (or any other IMovable subclass -- perhaps not the best example)? Testing the pointer against NULL each time you want to use it sounds a bit like a hassle to me. If that can't happen in your system, perhaps you could use a topological sort to find a dependence list between your components, and instantiate them in that order (e.g., create the RenderComponent before the PhysicsComponent since the latter needs the former). That way you wouldn't have to check if the reference has been created so often (I assume the Update() method is called in each iteration of the main loop). The topological sort approach could be problematic if your dependence graph contains cycles, however.

Quote:Original post by Hodgman
I've used systems in the past with this limitation, and it can get quite annoying in the long run (breaking entities into two entities for no reason...) I would recommend finding a work-around if possible.
Could you provide an example where an example where an entity might need multiple components of the same type? I'm curious to see situations where that problem arises. Does the usage of a multimap instead of a regular map (as I mentioned in my first post) sound like a reasonable solution?


Sphet, I like the idea of using a scripting language for the definition of entities. I could probably use an existing scripting language like Python or Lua instead of developing my own language/compiler/interpreter but even that sounds too big for my project. Thanks for the suggestion though; perhaps a scripting language will become necessary in the future after all.


Any other thoughts?
it seems rational thought has been killed and buried
As you mentioned, it seems like the central source of complexity here is the Mediator as a monolithic object. If all communication between components goes through the mediator, that can add up to a huge interface for the Mediator. Moreover, that interface is the union of all possible messages over all possible entities. A basketball's mediator will have to have a HandleCharactersHelmetHitByLargeFlightlessBirdEvent, because all of them do.
Quote:Original post by XezekielX
Testing the pointer against NULL each time you want to use it sounds a bit like a hassle to me.
No more of a hassle than using a central event manager ;)
Quote:If that can't happen in your system, perhaps you could use a topological sort to find a dependence list between your components, and instantiate them in that order (e.g., create the RenderComponent before the PhysicsComponent since the latter needs the former).
Speaking of dependance issues - these can be a real pain too. It might be worthwhile giving all components an EntityIsFullyConstructedNow() virtual function, so any initialization requiring other components can occur after all components have been created. These dependencies could be used as a sorting-order when calling this function, and no cycles should be able to exist (in debug you could check for cycles and assert/log the error).
Quote:Could you provide an example where an example where an entity might need multiple components of the same type? I'm curious to see situations where that problem arises. Does the usage of a multimap instead of a regular map (as I mentioned in my first post) sound like a reasonable solution?
Off the top of my head, we had a 'sprite' component for HUD-type stuff, and a Audio-source component for playback. Some UI elements required multiple sprites and/or sounds for a single 'logical entity', but had to be created as separate entities and tagged as a 'group' by the user.

Quote:Original post by Sphet
Hodgman, I am wondering what your interface registration system is like? I imagine that your templatized method Entity::GetInterface<T> must use T and extract a type-id to look it up - is it simply up to the child components to call Entity::RegisterInterface<T> so the same type-id can be used? Do you store it in a binary tree? I had thought to do a search across all the components ... This has the benefit of not requiring storage in the entity, but a linear traversal would require clients to hold onto the interface for performance reasons.
I have two ways of registering interfaces - static (compile time) registration, which uses macros to create a marshalling function - and dynamic registration, which uses a std::map within the entity.
The marshalling technique was suggested to my by Antheus in this thread, I can post more details about it when I'm at home. Basically it's a linear search through each component, but there is no storage overhead in the component or entity. As you suggest, components usually store the returned pointers in member variables to avoid searching every frame.
[EDIT]Also, type-id comparisons are horrendously slow with MSVC++, so I convert them to uint32's before doing any searching/comparisons with them. IIRC it looks something like:
template<class T> uint32 GetHash(){  static uint32 hash = Fnv32a( typeid(T).name() );  //debug code to record hash+name and check for collisions...  return hash;}
Quote:Original post by Hodgman
Speaking of dependance issues - these can be a real pain too. It might be worthwhile giving all components an EntityIsFullyConstructedNow() virtual function, so any initialization requiring other components can occur after all components have been created. These dependencies could be used as a sorting-order when calling this function, and no cycles should be able to exist (in debug you could check for cycles and assert/log the error).


Our system has three stages for components: 1. constructor, where the defaults are set without any knowledge of your peers/siblings - here components register themselves with subsystems; 2. Component::OnCreate - by this time any per-instance changes to a component instance have been set from source data files as defined in our level editor. 3. Component::OnPostCreate - by this time all of the components in an entity have been initialised, at least internally to themselves, and are now considered valid to their siblings. This solves all of our problems, and we are pretty satisfied with how it works.

Quote:Could you provide an example where an example where an entity might need multiple components of the same type? I'm curious to see situations where that problem arises. Does the usage of a multimap instead of a regular map (as I mentioned in my first post) sound like a reasonable solution?


We have multiple instances of some components where we want to expose multiple prop states in our UI. It just works/makes sense having support for more than one instance. Our components are stored in a table of offsets that is shared among each instance of the same 'pattern' of components. We don't allow at-runtime composition. Instead it is defined and performed in the pipeline, so much like a single virtual function table is shared among all instances of a class, a single instance of the patterns' component table is shared among all instances of a pattern. This lets us put a lot of rich data into a shared structure, minimizing memory usage while maximizing verbosity.

Quote:I have two ways of registering interfaces - static (compile time) registration, which uses macros to create a marshalling function - and dynamic registration, which uses a std::map within the entity.
The marshalling technique was suggested to my by Antheus in this thread, I can post more details about it when I'm at home. Basically it's a linear search through each component, but there is no storage overhead in the component or entity. As you suggest, components usually store the returned pointers in member variables to avoid searching every frame.


Sounds familiar to what we do - in fact, today I went and implemented GameEntity::FindInterface<T>() method, and used simple macros to define static hash IDs for our interfaces:
class ITransform{   DECLINTERFACE( ITransform )   virtual Matrix4& GetTransform() = 0;};

where DECLINTERFACE is
#define DECLINTERFACE(INTERFACE) static u32 GetInterfaceID() { static u32 hash = HashString( #INTERFACE ); return hash; }


Allowing us to do
template<typename T>T* GameEntity::FindInterface(){  FindInterfaceByID( T::GetInterfaceID( ) );}

I've expanded this to actually return an InterfaceIterator<T> so we can iterate over multiple interface implementations. The game teams are really excited about the abstraction provided.


XezekielX - as far as testing the pointer against null - that's one of the things you have to get used to. Instead of testing it against a NULL, consider it more like "If (HasAnimation) { do something }" instead of "if (pAnimation != NULL ) { do something }".

Also, components should not have a virtual Update() method. You do not want to do foreach( Entity ) foreach ( Component in entity ) - it's too much work thats not needed, since more components will have an empty Update() method. Instead you want to do this:
AnimationComponent::AnimationComponent(){   Animator::RegisterComponent(this);}

and in the Animator, invoked from the right place in your loop:
void Animator::UpdateAnimations( float deltaT ){   foreach( RegisteredComponent component )   {      component->UpdateAnimation( deltaT );   }}


This way you only update animation components, not all components. You can also make your animation component remove itself from the list if the animation is inactive, say in AnimationComponent::Stop(), and add itself back in through AnimationComponent::Play() - this way you only touch, per frame, the objects that need to be updated. Invoking a host of empty Update() methods is a waste of cycles, and with many entities, each of which as multiple components, the empty virtual call starts to add up - at least in my console world.

Good conversation, interesting stuff - keep it up.

.S

[Edited by - Sphet on June 10, 2009 9:27:25 AM]
One thing's for sure, ask ten people how to implement a component system and you'll get eleven (possibly good) answers. Everyone is going to be a proponent of what's worked for them in the past, however keep in mind that what has worked for others might not work or be appropriate for you.

The main thing to keep in mind when designing a component system, or any system for that matter, is exactly what you need your system to do. Start with the most important feature and go from there. You'd be surprised how many details fall out from examining the bare necessities.

Take the following XML snippet that defines several entities as well as the components they should have. This is similar to what our component system at work needs to handle (of course in real life it's a lot more complex than this):
<EntityType Name="TankA">   <PositionComponentType/>   <NameComponentType/>   <HealthComponentType>      <MaxHealth>100</MaxHealth>   </HealthComponentType></EntityType><EntityType Name="TankB">   <PositionComponentType/>   <MeshComponentType>      <Mesh>tank.x</Mesh>   </MeshComponentType>   <HealthComponentType>      <MaxHealth>200</MaxHealth>   </HealthComponentType></EntityType>


Assume that the above XML should roughly translate to the following C++ (in terms of data storage):
struct TankA{   // As a result of HealthComponentType   // For argument's sake assume this is legal   static int MaxHealth = 100;   // As a result of HealthComponentType   int Health;   // As a result of NameComponentType   std::string Name;   // As a result of PositionComponentType   Vector3D Position;};struct TankB{   // As a result of HealthComponentType   // For argument's sake assume this is legal   static int MaxHealth = 200;   // As a result of MeshComponentType   // For argument's sake assume this is legal   static const char* Mesh = "tank.x";   // As a result of HealthComponentType   int Health;   // As a result of PositionComponentType   Vector3D Position;};


Finally, you should be able to create instances of these entities using something similar to this:
Entity* e = ObjectSystem.Create_Instance_Of("TankA");IHealth* h = e->Get_Interface<HealthComponent>();IHealthType* ht = e->Get_Type()->Get_Interface<HealthComponentType>();h->Set_Health(ht->Get_Max_Health());

If you can achieve that, then congratulations, you have the core functionality of a component system! You still have to deal with issues like serialization, but in my experience the data-driven aspects of a component system are by far the hardest and trickiest to understand and get working, so if you can get those done and out of the way you'll have a large portion of the system designed and finished.
Quote:Original post by Sneftel
As you mentioned, it seems like the central source of complexity here is the Mediator as a monolithic object. If all communication between components goes through the mediator, that can add up to a huge interface for the Mediator. Moreover, that interface is the union of all possible messages over all possible entities. A basketball's mediator will have to have a HandleCharactersHelmetHitByLargeFlightlessBirdEvent, because all of them do.
I was looking forward to reading your opinion on my approach ;) I think I will be able to keep the events fairly generic and avoid things like HandleCharactersHelmetHitByLargeFlightlessBirdEvent but yes, I suppose it will eventually become a problem. Until then I think I will keep this approach since it suits my needs and offers other benefits.


Quote:Original post by Sphet
Also, components should not have a virtual Update() method. You do not want to do foreach( Entity ) foreach ( Component in entity ) - it's too much work thats not needed, since more components will have an empty Update() method.
Actually, my components don't share a common interface so there's no need to call an Update() method on each one of them. Some subsystems still call Update() on all components (PhysicsSubsystem, or Render() in the case of the RenderSubsystem) while others don't call anything (HealthComponents, for instance, only react to events, they don't need to be updated continually).

Quote:Original post by Sphet
*** Source snippet removed ***
You can also make your animation component remove itself from the list if the animation is inactive, say in AnimationComponent::Stop(), and add itself back in through AnimationComponent::Play() - this way you only touch, per frame, the objects that need to be updated.
Thanks for the suggestion, the AnimationComponent/Subsystem is actually an excellent example of where this might be handy.


Quote:Original post by Zipster
Finally, you should be able to create instances of these entities using something similar to this:
Entity* e = ObjectSystem.Create_Instance_Of("TankA");IHealth* h = e->Get_Interface<HealthComponent>();IHealthType* ht = e->Get_Type()->Get_Interface<HealthComponentType>();h->Set_Health(ht->Get_Max_Health());

If you can achieve that, then congratulations, you have the core functionality of a component system! You still have to deal with issues like serialization, but in my experience the data-driven aspects of a component system are by far the hardest and trickiest to understand and get working, so if you can get those done and out of the way you'll have a large portion of the system designed and finished.
The way I intended to create my game entities was to combine the EntityBuilder I presented in my first post and an XML file containing the definition of the entities:
<GameEntities>  <Entity name="Ship1">    <RenderComponent>      <Sprite img="ship1.png" />    </RenderComponent>    <PhysicsComponent>      <Size w="50" h="50" />      <Position x="0" y="0" />      <Velocity vx="100" vy="100" />    </PhysicsComponent>    <HealthComponent>      <Max value="100" />    </HealthComponent>    <!-- other components -->  </Entity>  <!-- other entities --></GameEntities>


The entity creation loop would look like this:
RenderSubsystem render;PhysicsSubsystem physics;HealthSubsystem health;// ...EntityBuilder builder;builder.RegisterSubsystem("RenderComponent", &render);builder.RegisterSubsystem("PhysicsComponent", &physics);builder.RegisterSubsystem("HealthComponent", &health);// ...// load the XML documentforeach(entity in entities){  builder.BuildComponent(entity->GetAttribute("name"));  foreach(node in entity)  {    builder.AddComponent(node->GetName(), node);    // GetName() would return the name of the XML tag  }}

The builder maps a string to a subsystem pointer, which is then used by the AddComponent() method (it calls the Create() method on the correct subsystem using the name of the tag, which corresponds to the registered name of the subsystem). The subsystem (or the component, it doesn't really matter) is then responsible for the extraction of the parameters from the XML node it received. I think it's quite an elegant solution.
it seems rational thought has been killed and buried
Quote:Original post by XezekielX
The way I intended to create my game entities was to combine the EntityBuilder I presented in my first post and an XML file containing the definition of the entities:*** Source Snippet Removed ***

The entity creation loop would look like this:*** Source Snippet Removed ***
The builder maps a string to a subsystem pointer, which is then used by the AddComponent() method (it calls the Create() method on the correct subsystem using the name of the tag, which corresponds to the registered name of the subsystem). The subsystem (or the component, it doesn't really matter) is then responsible for the extraction of the parameters from the XML node it received. I think it's quite an elegant solution.

Looks good to me, but one thing you need to think about is how you intend to handle static data versus instance data. Every instance of "Ship1" has a maximum health of 100 as specified in XML, so it's wasteful to store that value more than once. The same could be said about the sprite name. Something like position and velocity need to be stored per instance of course.

Making a distinction between these two types of data is crucial for scalability.

This topic is closed to new replies.

Advertisement