Followers 0

# Outboard component-based entity system architecture

## 106 posts in this topic

The entity would be a container for data-components, i.e. a transformation component (position, orientation etc.), a visual component (mesh, textures etc.), ...
Subsystems take the role of behaviour-components but there's only a single instance. Entities are then processed in a loop which works on whatever list may be appropriate for the subsystem. Data that is needed per entity but only within the subsystem could be stored using the decorator pattern.

When you add an entity to the world, it is added to the relevant subsystems. Those are found by
1. explicitly defining where it should be added (e.g. physics, visual, ai, ...). If the entity doesn't have the data required by a subsystem it either responds with some error or - if it can - creates the data-components and adds them to the entity.

2. iterating through each subsystem where each subsystem checks the entity for the required data-components and adds the entity to its list if all required data-components are found.

A data-component may have a list of observing subsystems but subsystems are not required to register to the data-components. Thus, if necessary, when a data-component is changed the registered subsystems are notified and may process the entity as the wish (e.g. add it to a "to-be-processed" list). Subsystems that would process each entity anyway would not have to register to the component but be able to poll it when needed.

Each frame the subsystems could run in order, i.e. ai->physics->visual. As said above each subsystem would process all the relevant entities (e.g. from the "to-be-processed" list), much like a particle system would do.

So, what do you say?
2

##### Share on other sites
Quote:
 Original post by Lord_EvilSo, what do you say?

I think I like this approach. So basically, entities are pure containers for components, data structures which only have data, no logic. Then, "subsystems" are the logic portion of the system. They are given entities, and perform operations on those entities, utilizing one or more of the data components in each entity to update the state of the system. A subsystem ignores an entity if it does not contain all components that it requires.

The beauty of this approach is that communication between components is not required at all. The subsystems are responsible for this communication. I.e. the graphics subsystem updates the position and orientation of the graphics component from that of the transformation component.

Logic could therefore be abstracted into a logic subsystem which perhaps has a scripting engine interface. Therefore each entity could have it's own logic component which would contain the script to be executed for logic updates for that entity. The script, since it is executed by the subsystem would have access to all of the components, even itself.

Genious, pure genious.
0

##### Share on other sites
Quote:
Original post by Sneftel
Quote:
 If you wanted an entity without a ScriptComponent for example, then how does the AnimationComponent detect this so that it can feed its data into the UserInputComponent instead?
I'm not sure what you're asking here... though I also have no idea what AnimationComponent would have to say to UserInputComponent.

"At this point in the current animation, if the user does a specific action, it triggers a specific response" (for example, clicking at the right time when fighting in The Witcher allows the player to make a combo).

I'm going to read that thread from the beginning to its end - it's quite long, but it's quite interesting too. I hope I'll be able to contribute something of interest [smile].
0

##### Share on other sites
Quote:
 Original post by AresseraGenious, pure genious.

Erm ... well, that's too much of praise for me [embarrass]
0

##### Share on other sites
Quote:
 Original post by Lord_EvilSubsystems take the role of behaviour-components but there's only a single instance. Entities are then processed in a loop which works on whatever list may be appropriate for the subsystem.

The primary reason that I kept components separate in my design is the Single Responsibility Principle. This sounds like it would work fine.
Quote:
 Data that is needed per entity but only within the subsystem could be stored using the decorator pattern.

What do you mean here? The decorator pattern isn't great at adding heterogeneous data to an object.
Quote:
 A data-component may have a list of observing subsystems but subsystems are not required to register to the data-components. Thus, if necessary, when a data-component is changed the registered subsystems are notified and may process the entity as the wish (e.g. add it to a "to-be-processed" list). Subsystems that would process each entity anyway would not have to register to the component but be able to poll it when needed.

This is a very interesting approach. I'd never thought about listening on data-component events, only on entity events.
Quote:
 So, what do you say?

It's a very clever approach. I'll have to think more on it.
0

##### Share on other sites
Quote:
Original post by Emmanuel Deloget
Quote:
Original post by Sneftel
Quote:
 If you wanted an entity without a ScriptComponent for example, then how does the AnimationComponent detect this so that it can feed its data into the UserInputComponent instead?
I'm not sure what you're asking here... though I also have no idea what AnimationComponent would have to say to UserInputComponent.

"At this point in the current animation, if the user does a specific action, it triggers a specific response" (for example, clicking at the right time when fighting in The Witcher allows the player to make a combo).

Ah, I got it. In that situation, the component would ask the other subsystem whether it had a component for that entity. Or the flow of control would be inverted, and an event would be raised for anyone who cared.
0

##### Share on other sites
Quote:
Original post by Sneftel
Quote:
 Data that is needed per entity but only within the subsystem could be stored using the decorator pattern.

What do you mean here? The decorator pattern isn't great at adding heterogeneous data to an object.

Well, maybe decorator isn't the correct term. I mean something like:
class PhysicsEntity{  private:    EntityPtr entity;    Box       boundingbox;      public:    ...}

I'd just do this to add data that isn't necessarily stored somewhere but might be used by the subsystem. In the example given you won't store a bounding box within the entity but since it's needed by the physics system you'd calculate the box when you add the entity and store it within the physics subsystem. If multiple subsystems would make use of the bounding box it should be promoted to be part of some data-component. (Maybe a bounding box is not the ultimate example here [smile].)

I could also keep some other data structure along with the entities but that would be error prone (e.g. when removing the entity but not the additional data) and ugly.
Quote:
Original post by Sneftel
Quote:
 So, what do you say?

It's a very clever approach. I'll have to think more on it.

I'd appreciate your thoughts on this. I was inspired by your approach and the discussion of it made me think of how I could design my engine's component model for weeks.
0

##### Share on other sites
Quote:
 Original post by Lord_EvilWhat about this approach:The entity would be a container for data-components, i.e. a transformation component (position, orientation etc.), a visual component (mesh, textures etc.), ... Subsystems take the role of behaviour-components but there's only a single instance. Entities are then processed in a loop which works on whatever list may be appropriate for the subsystem. Data that is needed per entity but only within the subsystem could be stored using the decorator pattern.When you add an entity to the world, it is added to the relevant subsystems. Those are found by 1. explicitly defining where it should be added (e.g. physics, visual, ai, ...). If the entity doesn't have the data required by a subsystem it either responds with some error or - if it can - creates the data-components and adds them to the entity.2. iterating through each subsystem where each subsystem checks the entity for the required data-components and adds the entity to its list if all required data-components are found.A data-component may have a list of observing subsystems but subsystems are not required to register to the data-components. Thus, if necessary, when a data-component is changed the registered subsystems are notified and may process the entity as the wish (e.g. add it to a "to-be-processed" list). Subsystems that would process each entity anyway would not have to register to the component but be able to poll it when needed.Each frame the subsystems could run in order, i.e. ai->physics->visual. As said above each subsystem would process all the relevant entities (e.g. from the "to-be-processed" list), much like a particle system would do. So, what do you say?

If I understand everything, subsystems are made of a query (to get the entities they can work on) and a procedure (to apply a function on the entities). It seems to be simple and elegant and do not assume any kind of scene structure (do you can use a scene graph or any other world management structure).

In fact, it looks like a database to me - which is a good point imho. I would go without the observer mechanism as it introduces out-of-order processing (something which is difficult to manage when you have a lot of data). If you can cope with that then it's ok.

Question: how do you manage
1) data duplication (the bounding box can be of interest to various components; should all components have an embedded bounding box)?
2) components requirements (because I have this component, I shall also have this other one)?
0

##### Share on other sites
Quote:
 Original post by Emmanuel DelogetQuestion: how do you manage 1) data duplication (the bounding box can be of interest to various components; should all components have an embedded bounding box)?2) components requirements (because I have this component, I shall also have this other one)?

Maybe these both questions could be answered by the same mechanism, when adding a component to an Entity?
A subsystem which depends on/Observes a different subsystem will most likely know which component(s) the other/Subject subsystem has, say a bounding box. It would not be much point for a subsystem to depend on another one unless it knows what the other one has/does, right? Atleast to some degree...

Thus, if subsystems try to intitialize or add its component to an Entity, before it has another component it depends on, it should know which component needs to be added by the missing component's owning subsystem. So it could simply (maybe recursively) tell that other subsystem to init the Entity before it does so itself.

To avoid duplicated data, a subsystem could also obtain direct references to other components of that Entity, or perhaps even members in those components, during the add-component/init phase. The references would be relayed to (and stored in) the component which needs them to function by its owning subsystem.
There should be little problem as long as subsystems which rely on components from other subsystems make sure they are added before their own.

This hopefully also avoids having to "manually" update all members in a component which relies on other component's members, as they would automatically point to updated data at all times.

To avoid making these "couplings" so hard that there'll be trouble when changing members of a component/subsystem which are relied upon by other subsystems/components, one could use ideas from Game Programming Gems 2, "A Property Class for Generic C++ Member Access" so exposed component or subsystem members could be referenced indirectly by a more descriptive name such as

Of course, this "property lookup" between components should all be done during Entity init to avoid performance drops due to having to go through a bunch of indirect couplingd.

What do you all think?
0

##### Share on other sites
Quote:
 Original post by Lord_EvilI'd just do this to add data that isn't necessarily stored somewhere but might be used by the subsystem. In the example given you won't store a bounding box within the entity but since it's needed by the physics system you'd calculate the box when you add the entity and store it within the physics subsystem.

This starts sounding a lot like my approach, where subsystem-specific data is stored by the outboard component. Rename PhysicsEntity to PhysicsComponent and you'll see what I mean.

I agree with Emmanuel-- it's starting to sound a lot like a database. Which isn't a bad thing. Of the mainstream software development ideas which game development has grudgingly absorbed--OOP, data-driven architecture, scripting integration--I really think that RDBMSes are next. I disagree with him that the observers are a bad thing, though. Just remember your class invariants, and make sure that they're satisfied whenever you invoke an observer. Do that, and the screwed up control flow won't matter.
0

##### Share on other sites
Quote:
 Original post by Emmanuel DelogetQuestion: how do you manage 1) data duplication (the bounding box can be of interest to various components; should all components have an embedded bounding box)?

Well, if some data is potentially used by multiple subsystems I'd create a corresponding component.

Quote:
 Original post by Emmanuel DelogetQuestion: how do you manage 2) components requirements (because I have this component, I shall also have this other one)?

I'm not sure if I fully understand that question but if multiple components are needed the subsystem would check for those components individually and either reject the entity or create a default component (a bounding box, for example, could be calculated from other spatial data).

Other components could be optional. As an example the visual subsystem could expect the mesh but could also make use of some shader component.

Edit: I'd not use the Observer mechanism to change the order of processing but to build lists of entities that have changed. Then when it's the subsystem's turn within the processing order it will remember what entities/components have changed and only process those.

[Edited by - Lord_Evil on October 30, 2007 2:14:08 PM]
0

##### Share on other sites
I've been thinking of moving to "outboard" component storage and find myself with a pretty much 1:1 ratio of subsystems to component types. I can see myself having many, many component types. How do you manage all these? Also, this seems to make it a lot more difficult to create new components, especially from a script.

On the otherhand I see problems with "inspecting" each entity for various components that require attention each frame. Are there any other solutions? I was thinking of adding a "onTick" global type event that components can hook into which would let them get updated/drawn/whatever each frame without requiring iteration through a list of entities. Any thoughts?
0

##### Share on other sites
Quote:
 Original post by wild_pointerI've been thinking of moving to "outboard" component storage and find myself with a pretty much 1:1 ratio of subsystems to component types. I can see myself having many, many component types. How do you manage all these? Also, this seems to make it a lot more difficult to create new components, especially from a script.

I see this too in a way. I'm currently sketching out an engine design based on what's been said in this thread and other places, and there does seem to become a 1:1 (or near) relation between components and subsystems. Of course you could allow a subsystem to add more components to an Entity (or just associate them with the Entity and store them in the system itself), depending on which behavior it should have.

I guess this depends on how exactly you define your components. On the Engine-level (no game specific components here) I see few occasions where I would need more than one component at a time from each subsystem. It might be a different component depending on type of animation needed from the Animation subsystem and such, but I don't think one would need more than one component from that specific system.

On this level subsystems seem to become very strongly associated with their own components. So if you want to add a new behavior to your Entity, you might need to write a new subsystem to take care of that specific task. (Especially since components here seem more like data containers as I've understood it.) #1

I'm going for a subsystem interface, so I can add new subsystems at any time and know that the engine knows how to pass it an Entity if told to (from script perhaps).

This could also mean some game specific (above engine level to be clear) subsystems are implemented in script, utilizing script implemented components. The subsystem could of course also be dynamically linked if one so wishes.

EDIT:
#1) Hmm, what if one would stretch the component "pattern" further and include "plug-and-play behavioral components" in say the animation system? If I want a different type of animation behaviour (skeleton instead of frame based), but would like to keep the framework of the animations subsystem, I could make a new behavior component for the subsystem, and a data component for the Entity. I'm not sure if this would be practical at all, it just hit me it might be worth a thought or two... it's probably more work than replacing the whole animations subsystem with one that does all you want to tho.

[Edited by - TwoD on November 1, 2007 10:30:30 AM]
0

##### Share on other sites
i've been using a data-driven design for a while now which im not happy with heres the gist of it:

class Message{}class Entity{  bool HandleMessage(Message* msg);  array<Entity*> children;  Entity* parent;}

think of the top most level as an entity, then all of its children and thier children are components.

the problem i had with this is that communication is a problem between entitys.

many times i would just assume my parent is the entity i needed communication with.

All communication must be done using HandleMessage. This is also another problem. Its ugly having hundreds of messages, but it does make for some cool scripting opportunities!

i've been thinking about this stuff alot, and my new system im about to prototype after reading other peoples thoughts and scouring the web for what others have done is this:

class Entity{  InsertComponent(component)  RemoveComponent(component)  GetComponent(type)  array<Component*> components}class Component{  InsertObserver(component)  RemoveObserver(component)  NotifySubjectChanged(component)  NofityObservers()  Entity* owner}

the observer pattern looks like it can solve my communication problem, but this means components need to know about other components, which i think is fine so far here's an example i came up with:

class HealthComponent{  void TakeDamage(damage)  {    health -= damage    NotifyObservers()  }  void Regenerate(amount)  {    health += amount    NotifyObservers()  }  float health}class ScoreComponent{  void Init()  {    HealthComponent* hc = owner->GetComponent(HealthComponent::Type)    hc->InsertObserver()  }    void NotifySubjectChanged(subject)  {    switch (subject->GetType())    {    case HealthComponent::Type:      HealthComponent* hc = subject;      if (hc->health <= 0)        deaths++;      break    }  }  int deaths;  int kills;  int score;}

creating entitys is easy to do:
Entity e;e.InsertComponent(new HealthComponent);e.InsertComponent(new ScoreComponent);e.Init();

Components also act as interfaces, so i dont need the ugly messages, i can do this:

HealthComponent* hc = e.GetComponent(HealthComponent::Type);if (hc)  hc->TakeDamage(10.f);

im just wondering how to do state machine type code with this system now, and how im going to do scripting. Might need a special function
HandleScriptCommand as part of a component
0

##### Share on other sites
If anyone could post some guidelines about how to divide components and how to use this paradigm using a script system I would be really grateful.

And by the way I will make a really newbie question regarding this system.

Think of an entity that has a Component containing its position, orientation, etc... And a Physics component. The Physics component needs to know the position of the entity, how do you retrieve it? I have been read through the thread about sending messages, etc... wouldn´t be enough doing something like $this->Owner->Get Component("PositionOrientationComponent")->GetPosition()?. And isn´t this a lot of trouble to just get the position every time it is needed?. Sorry for the newbie question... Thanks in advance, HexDump. [Edited by - HexDump on November 20, 2007 10:30:31 AM] 0 #### Share this post ##### Link to post ##### Share on other sites Quote:  Original post by HexDumpAnd by the way I will make a really newbie question regarding this system.Think of an entity that has a Component containing its position, orientation, etc... And a Physics component. The Physics component needs to know the position of the entity, how do you retrieve it? I have been read through the thread about sending messages, etc... wouldn´t be enough doing something like$this->Owner->Get Component("PositionOrientationComponent")->GetPosition()?. And isn´t this a lot of trouble to just get the position every time it is needed?.

I believe Sneftel mentioned that the entity actually has spatial information, because many subsystems needs it. The outboard components knows the entity and can therefore get the positioning info directly from it. He could have made a SpatialComponent, but that will only make almost all of the subsystems depend on that instead.

0

##### Share on other sites
dingojohn, this was just an example, what I wanted to know if this is the way components get things from another components.

As I said in my last post if anyone could explain some guidelines about how to divide components and how to use this paradigm using a script system I would be really gratefull.

By the way, what are this subsystems you´re talking about? render system? input system? IA system? etc...?

HexDUmp.
0

##### Share on other sites
Quote:
 Original post by notbaddingojohn, this was just an example, what I wanted to know if this is the way components get things from another components.

If a subsystem knows about another subsystem, then it can find out about the other subsystem's components. That lookup information is necessary anyway (for when entities are removed). So when a component is created which will depend on another component, the creation process can resolve that linkage. From there on out, the component can just ask the other component.

I can't really give you great guidelines on how to divide components, other than to say that this sort of an architecture leads to slightly fewer components than the standard onboard component system tends to. As for scripting, that really depends on where scripting fits into your game architecture.

Quote:
 By the way, what are this subsystems you´re talking about? render system? input system? IA system? etc...?

The first and third are good matches. For the input system, since only one entity is controlled, the setup can be a lot more simple.
0

##### Share on other sites
I´m using LUA, if this is of some information.

HexDump.
0

##### Share on other sites
Quote:
 Original post by HexDumpI´m using LUA, if this is of some information.

A little bit, but not much. In terms of control flow, information flow, etc., how does scripting fit into your game architecture?
0

##### Share on other sites
Well, I don´t really know what you mean, but I can tell you I´m using couritines for my entitnes (each thread is a entity). I Have some functions that define the behaviour of the entity, I mean, Events, like OnHit(), OnDie(), OnCreate(), etc...

From my entity system I create a lua entity (it is a C++ class) that calls the OnCreate() in lua and this sets everything the entity needs to init (mesh used, speed, etc...), after it comes back from OnCreate(), I finish the initialization in C++.

Sorry if I´m making silly questions, but I really afraid to move my design to component system and having lot of undercovered problems.

By the way, what are this events system used for? I read something about communication between components? I think there's a more direct way to do this, don´t you think?.

The other problem I see with component system is the granurality of it. I think we will end up with hundreds of components for just everthing.

HexDump.
0

##### Share on other sites
Well, I have been doing some drawings and thinking about my design :). Here is where I am so far, I hope could give his opinion about it:

1) I will go with a component system where entities will be a container for the components.
2) The components will be stored in a map/hash, with key "component name", instance.
3) When a component is added to an entity, it will publish to the entity callbacks
to handle the component, example:

====> $this->Owner->RegisterAction("UPDATE",UpdateCallback); ====>$this->Owner->RegisterAction("GOTO",GotoCallback);
====> ....

This registrations could be mantained in a map/hash of type <String,CallBack>
A call for example to get the Position of a entity (if we have decided to to put this in the entity itself because of its use in almost all entities) to a Frame componenet would be something like:

====> \$this->Owner->DoAction("GETPOSITION");

and this will give us position of the entity.

This provides me with a mecahnism to not having hundreds of simple methods in entity that the only thing that do is delegating to the real method of the component.

4) In order to handle parameteres, returns, we could use a ParamList that will be passed, or returned by the Action.

What do you think about it?.

HexDump.
0

##### Share on other sites
This is a great thread, I've learned a lot. Inverting the component system is so beautifully simple I can't believe more people haven't thought of it and been advocating it.

One thing though, I'm not sure this system lends itself well to threading across many, many cores. It would be easy enough to throw each subsystem into it's own thread and let them run asynchronously, but eventually I can't see you coming up with enough subsystems for all the cores you will have. And since objects are just data containers in this case, with no reference back to their components except loosely through listeners, it would be difficult to group spatially close objects and throw those at threads instead.

Any thoughts?
0

##### Share on other sites
Quote:
 Original post by ThergothonOne thing though, I'm not sure this system lends itself well to threading across many, many cores. It would be easy enough to throw each subsystem into it's own thread and let them run asynchronously, but eventually I can't see you coming up with enough subsystems for all the cores you will have. And since objects are just data containers in this case, with no reference back to their components except loosely through listeners, it would be difficult to group spatially close objects and throw those at threads instead.

Actually, I think this system is probably more amenable to parallelization than the standard component design. If you can't come up with enough subsystems to give all your cores something to do, just split a subsystem update across multiple cores. That's probably a good idea regardless, since different subsystems will eat up different-sized CPU slices. (This is assuming your subsystem updates can be parallelized; if they can't, you're screwed regardless of which design you go with). The key here, I think, is a job-based as opposed to a thread-based model, which Sony's been trying to convince people to go with anyway.
0

##### Share on other sites
Makes sense, looks like I'm going to have to hit up google for some job-based v. thread-based articles.

Cheers.
0

## Create an account

Register a new account