Jump to content

  • Log In with Google      Sign In   
  • Create Account


Component based architecture and messaging


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
19 replies to this topic

#1 Dominik2000   Members   -  Reputation: 249

Like
4Likes
Like

Posted 09 September 2013 - 05:28 AM

Hello,

 

I have a component based architecture with subsystems, which hold the component data and implement the game logic. Now a great problem is, how to implement a messaging system.

 

I have a message dispatcher system, with template where every message type registers it's own eventhandler. On Dispatch every function in the list will be called. I don't like this, because it's hard to delete one listener and I don't get how a message queue could be implemented.

 

So I think of a design like that every subsystem has it's own queue and a high level system (e.g. world manager, ...) write to this queue if anything needs to be updated. One problem is, I think, that every high level system needs to know about the low level systems.

 

Is this a good approach? How do you think about coupling/decoupling, because coupling would be the case in my idea.

 

Dominik



Sponsor:

#2 Dennis Technologies   Members   -  Reputation: 147

Like
1Likes
Like

Posted 09 September 2013 - 09:50 AM

Most people will tend towards decoupling out of efficiency, being easier to change components. Coupling can work, and can (I stress 'can') be faster to implement as well as be faster overall, however your system will be less robust and will be far more fragile.

Your second idea does not require 'coupling' as I would understand it. Have you looked at Service Oriented Architectures? I think it would fit well and is a decoupled system.

#3 crancran   Members   -  Reputation: 373

Like
5Likes
Like

Posted 09 September 2013 - 10:01 PM

Don't over-engineer something which can be easily modeled in a simplistic fashion and changed later if the need arises.

 

The premise behind a component based architecture isn't to abolish coupling completely, but to hide that coupling effectively so that designers can easily combine a set of features exposed through an API to create unique and creative game objects with ease.  So what becomes important to us developers is how and where we implement the coupling.

 

The simplest of approaches would be to leverage the observer pattern; allowing systems to register as a listener to other systems.  Depending on your approach, this may lead to the massive switch statement event handler or too an excessive multiple inheritance problem.  I'd argue if you end up with either, chances are the system is likely doing too much and might suffice for being split into smaller chunks of behavior.  

 

Another approach would be to have a system expose a set of event objects that another system could subscribe/unsubscribe as needed.  These event objects are analogous to signal/slots which can be found in boost and several other libraries.  The benefit is that using signal/slots, the connection between the subscriber and publisher can be cleaned up when either go out of scope.  Function callbacks are wonderful constructs, but do be weary of their side affects, particularly on your cache if over used inside your main game loop.

 

One last approach would involve reversing the flow of data.  In the above two scenarios, you're pushing data from a higher level system to the lower systems, but sometimes it can be far more efficient and flexible to poll for information depending on circumstances.  

 

In this case, you have an event buffer that holds events your game loop has fired in the current frame sorted by time.  When a system processes it's update tick, it checks whether any events of interest exist in the event buffer and processes them.  This would allow you to skip events if multiple ones exist and it would also allow events to be popped from the event buffer and replaced with modified versions that perhaps are of interest to a system that fires later in the update cycle too (e.g. system that modifies damage event based on armor, shields, absorbs, etc.).  In this case, you can easily store the events placed in the buffer for later playback if that was something of interest.


Edited by crancran, 09 September 2013 - 10:05 PM.


#4 Dominik2000   Members   -  Reputation: 249

Like
1Likes
Like

Posted 10 September 2013 - 06:30 AM

I think, one great possibility is to go the third way, because every system can take the event that it can consume, but how can I implement this, without one big switch in every system. I think this is a very bad solution. Should I static_cast the event parameter?

In my event system (unbufferd) I have function calls, so it's easy to have the right parameter. But a solution with a big switch and a static_cast is simply ugly I think.

 

I think you have this event queue in the game loop class?



#5 Dominik2000   Members   -  Reputation: 249

Like
1Likes
Like

Posted 11 September 2013 - 05:42 AM

I have thought about a basic architecture, what I need in my game. I want a car to go to different points controlled by AI.

 

So I think I have following low level systems: Render system, Physic system (optional), Position system, AI system, Waypoint system.

High level system would be exactly one Entity sytem, which has references to all the low level systems, because it needs to know everything from above.

 

Is this correct? Is a component system structured this way?



#6 crancran   Members   -  Reputation: 373

Like
1Likes
Like

Posted 11 September 2013 - 10:56 PM

There are numerous ways to structure your events and they don't necessarily have to be classes.  Remember, one could easily just treat the event queue as a raw buffer of data and with some bookkeeping constructs, one could easily iterate the events perhaps by id within the raw buffer.  Events after all are just data, right? :)

 

The important thing to realize with a callback mechanism is that when you execute the callback immediately when the event happens, you often trash your instruction and data caches and this is particularly bad in games.  For example event A fires callback A.  While processing callback A, a condition is met that must fire event B that in turn fires callback B.  Inside callback B, a condition is met that must fire another event.  You can quickly see how your instruction/data caches can get trashed.

 

Just make sure that if you move forward with such an event system that it offers both asynchronous and synchronous invocations of your callbacks.  In the case of asynchronous, you queue up the events and a dispatch call at some future point issues all the callbacks in a loop where synchronous would fire the callback for a specific event at the time you emit the event.

 

You can place this queue where-ever is applicable for your code base.  In my engine, I have a framework class which exposes a number of core engine components through the framework interface since the framework interface is generally injected into all classes during initialization or construction.

 

 

 

I think, one great possibility is to go the third way, because every system can take the event that it can consume, but how can I implement this, without one big switch in every system. I think this is a very bad solution. Should I static_cast the event parameter?

In my event system (unbufferd) I have function calls, so it's easy to have the right parameter. But a solution with a big switch and a static_cast is simply ugly I think.

 

I think you have this event queue in the game loop class?



#7 crancran   Members   -  Reputation: 373

Like
1Likes
Like

Posted 11 September 2013 - 11:16 PM

I have thought about a basic architecture, what I need in my game. I want a car to go to different points controlled by AI.

 

So I think I have following low level systems: Render system, Physic system (optional), Position system, AI system, Waypoint system.

High level system would be exactly one Entity sytem, which has references to all the low level systems, because it needs to know everything from above.

 

Is this correct? Is a component system structured this way?

 

I'd say a bit of this is a matter of taste. 

 

There is no right or wrong answer on how to design a component system because while games are starting to shift and ship with some component system like concepts, it's still something that depending on the development team and architect who is at the helm of the design process; it all varies and sometimes significantly.

 

I generally prefer the idea that the entity framework is a core component to the engine.  The entity framework exposes an entity manager along with a component system interface.  The developers create derived classes of the component system interface for whatever logic they want to implement for the designers.  These component systems are then registered with the entity manager at start-up, allowing the entity manager to build a repository of meta data about the available components, properties associated with the components (useful for editors), etc.  

 

The beauty is that with the entity manager having all this metadata at it's finger tips, the engine simply exposes the entity manager to the remainder of the application.  Internally, the entity manager can delegate creation and destruction of components to the appropriate component systems without the developer having to find the right system and call it directly.  Our component system interface is also designed so that the component system can interact with our core engine framework and submit jobs to the task manager, register themselves as callback listeners to various systems, etc.



#8 Dominik2000   Members   -  Reputation: 249

Like
1Likes
Like

Posted 12 September 2013 - 06:12 AM

 

I have thought about a basic architecture, what I need in my game. I want a car to go to different points controlled by AI.

 

So I think I have following low level systems: Render system, Physic system (optional), Position system, AI system, Waypoint system.

High level system would be exactly one Entity sytem, which has references to all the low level systems, because it needs to know everything from above.

 

Is this correct? Is a component system structured this way?

 

I'd say a bit of this is a matter of taste. 

 

There is no right or wrong answer on how to design a component system because while games are starting to shift and ship with some component system like concepts, it's still something that depending on the development team and architect who is at the helm of the design process; it all varies and sometimes significantly.

 

I generally prefer the idea that the entity framework is a core component to the engine.  The entity framework exposes an entity manager along with a component system interface.  The developers create derived classes of the component system interface for whatever logic they want to implement for the designers.  These component systems are then registered with the entity manager at start-up, allowing the entity manager to build a repository of meta data about the available components, properties associated with the components (useful for editors), etc.  

 

The beauty is that with the entity manager having all this metadata at it's finger tips, the engine simply exposes the entity manager to the remainder of the application.  Internally, the entity manager can delegate creation and destruction of components to the appropriate component systems without the developer having to find the right system and call it directly.  Our component system interface is also designed so that the component system can interact with our core engine framework and submit jobs to the task manager, register themselves as callback listeners to various systems, etc.

 

 

I like your architecture and edited my architecture diagram to this:

http://i40.tinypic.com/2wggbuu.png

 

So I think, it's very similar to your description.



#9 crancran   Members   -  Reputation: 373

Like
1Likes
Like

Posted 12 September 2013 - 10:08 PM

I like your architecture and edited my architecture diagram to this:

http://i40.tinypic.com/2wggbuu.png

 

So I think, it's very similar to your description.

 

 

The only thing I'd caution you and possibly others about from your diagram is what you call Render System.  I often consider the rendering system the renderer, the system that interacts with OpenGL or DirectX.  The rendering system is typically something exposed by the engine itself much in the same light Physics, Input, Platform I/O, Networking, and Audio are exposed.  

 

Then you create specialized component subsystems you register with the entity framework that handle specialized behaviors.  This way you can have many component subsystems that perhaps handle specific rendering features, allowing each of those component subsystems to interact with the engine's rendering system.



#10 Dominik2000   Members   -  Reputation: 249

Like
1Likes
Like

Posted 13 September 2013 - 02:36 AM

I have edited the diagram, now I have a render manager, and a render system component. It's better I think, what I don't know is, if the waypoint system should be a system or manager. It' has notihing to do with the entity itself... Hmm....

 

http://i39.tinypic.com/oqg4sy.png



#11 BeerNutts   Crossbones+   -  Reputation: 2762

Like
1Likes
Like

Posted 20 September 2013 - 12:51 PM

I just wanted to ask if you've considering not using an event/messaging system?  Considering you have systems that will operate over components, and these components contain some data, why not have these "events" simply be a new component, or data in a component?

 

For whatever event you think an entity or component might need, you should be able to solve it using data in the component, or a specific component type.

 

For example, if you want to have a "collision event" why not create a collision component, and store it in the entities that are colliding.  then, when you run your collision system, it will run over those entity's with the collision component.

 

At least consider it.  I'm not saying you must use it, but I believe it to be a viable solution.


My Gamedev Journal: 2D Game Making, the Easy Way

---(Old Blog, still has good info): 2dGameMaking
-----
"No one ever posts on that message board; it's too crowded." - Yoga Berra (sorta)

#12 Reitano   Members   -  Reputation: 488

Like
3Likes
Like

Posted 25 September 2013 - 05:14 AM

In my engine, components are fully autonomous classes and do not share data with other components. For example, both the Transformation component and the RigidBody component have a vec3d position member variable. The problem is how to synchronize the components so that the position in the Transformation component is always an up-to-date copy of the RigidBody position. In turn, other components such as the MeshRenderer, Light, which all possess a position, needs a copy of the position of the Transformation component.

Often, message queues or callbacks are suggested. Instead, I use a simpler approach. All my component systems know the required dependencies and explicitly synchronize components once per frame. An example:
 

struct RigidBodySystem::SyncRigidBodyTransform
{
    const RigidBody* rigidBody;
    Transform*       transform;
};

RigidBodySystem::InitializeComponent(RigidBody* rigidBody, Entity* entity, Scene* scene)
{
    Transform* transform;
    if (scene->Query(entity, &transform))
    {
        // Add synchronization primitive
        this->syncPrimitives.push_back( SyncRigidBodyTransform(rigidBody, transform) )
    }
}

RigidBodySystem::DestroyComponent(RigidBody* rigidBody)
{
    RemoveSyncPrimitive(rigidBody)
}

// Called once per frame
void RigidBodySystem::Update(float dt)
{
    this->physicsEngine->Update(dt)
}

// Called every frame after Update
void RigidBodySystem::SynchronizeComponents()
{
    foreach (sync, this->syncPrimitives)
    {
        sync.transform->SetPosition(sync.rigidBody->GetPosition())
        // Orientation too...
    }
    
}

The synchronization phase is very efficient and cache friendly because the in and out components reside in consecutive memory areas and the loop does not trash the instruction cache.

An objection might be that this approach introduces coupling between classes. That is true but in my case the coupling happens at a high level. In fact, component systems like the RigidBodySystem are simple frameworks that dispatch calls to low-level libraries and these have no dependencies on other component libraries.



#13 phil_t   Crossbones+   -  Reputation: 3234

Like
0Likes
Like

Posted 25 September 2013 - 01:06 PM


Instead, I use a simpler approach. All my component systems know the required dependencies and explicitly synchronize components once per frame.

 

That's what I do too, and I think it's a reasonable approach. It's fine for that dependency to be encapsulated in the systems.

 

 

 


I have edited the diagram, now I have a render manager, and a render system component. It's better I think, what I don't know is, if the waypoint system should be a system or manager. It' has notihing to do with the entity itself... Hmm....

 

When you say "it has nothing to do with the entity itself", it makes me wonder if you fully understand the point of an entity-component-system framework.

 

In my view, all game objects should be entities. Not all entities have to have the same set of components - in fact most entities will just use a few. The waypoint has nothing to do with the player entity, say. But it is its own entity. I would think you would have a waypoint entity that is comprised of a Position and Waypoint component (and a Render component?). Or possibly the Waypoint component could describe a full series of waypoints itself (that would make it less trivial for them to be rendered, but then you wouldn't need some other component/entity that is responsible for connecting waypoints together in a path.

 

In your diagram, I would also question the lack of an InputSystem. Surely entities will need some sort of Component that describes how they are controlled - e.g. what input they respond to. So just like you have a RenderComponent and RenderSystem, you might want an InputComponent and InputSystem in addition to your InputManager which performs more low level/platform-specific tasks.

 

Also, you have a box called "Logic" above your engine. What do you envision going in there? I think the bulk of game logic should be in the systems or in individual scripts attached to the entities.


Edited by phil_t, 25 September 2013 - 01:07 PM.


#14 Dominik2000   Members   -  Reputation: 249

Like
0Likes
Like

Posted 30 September 2013 - 07:19 AM

I like the idea to remove the event system. The sync of the position between the components was the problem, and the reason to not remove the event system. I think I understand Reitanos approach.

 

 

 


In my view, all game objects should be entities. Not all entities have to have the same set of components - in fact most entities will just use a few. The waypoint has nothing to do with the player entity, say. But it is its own entity. I would think you would have a waypoint entity that is comprised of a Position and Waypoint component (and a Render component?). Or possibly the Waypoint component could describe a full series of waypoints itself (that would make it less trivial for them to be rendered, but then you wouldn't need some other component/entity that is responsible for connecting waypoints together in a path.

 

Yes but I want to have one place to store all waypoints and pathes to other waypoints. In my opinion every entity has it's own waypoint component. So I have much duplicate data in store...

 

 

 


In your diagram, I would also question the lack of an InputSystem. Surely entities will need some sort of Component that describes how they are controlled - e.g. what input they respond to. So just like you have a RenderComponent and RenderSystem, you might want an InputComponent and InputSystem in addition to your InputManager which performs more low level/platform-specific tasks.

 

Yes I don't have drawn the Input system, sorry for this. And how should I do this without event messaging? How could I exit the main loop?

 

 

 


Also, you have a box called "Logic" above your engine. What do you envision going in there? I think the bulk of game logic should be in the systems or in individual scripts attached to the entities.

 

E.g. I don't know where to put my money system. This I would place in the logic. Or is this another system?


Edited by Dominik2000, 30 September 2013 - 07:19 AM.


#15 phil_t   Crossbones+   -  Reputation: 3234

Like
0Likes
Like

Posted 30 September 2013 - 03:37 PM


E.g. I don't know where to put my money system. This I would place in the logic. Or is this another system?

 

What do you envision your money system doing? It's hard to give an opinion without knowing what functionality you require.

 


Yes I don't have drawn the Input system, sorry for this. And how should I do this without event messaging? How could I exit the main loop?

 

I'm sorry, I'm not really sure what you're asking here. During your update tick, your InputSystem should be able to query what key/mouse/gamepad events have taken place since the last tick.

 


Yes but I want to have one place to store all waypoints and pathes to other waypoints. In my opinion every entity has it's own waypoint component. So I have much duplicate data in store...

 

You really should avoid any duplicate data. Without knowing more about how your waypoints are created/used/modified (are they static points on a map? Is the player interacting with the game to create them? Is the AI using them to move game objects along a path?), it's hard to offer advice. You could certainly make your way-points completely separate from your entity-component-system architecture - if that makes sense. But if you find, say, that you're suddenly writing a ton of code to synchronize the game entities with your waypoint manager, then maybe it makes more sense to have them tightly integrated with your ECS.

 

Try listing out the various scenarios involving waypoints. For instance: "the user adds a waypoint to his character's path with the mouse", "some way-points are drawn as floating white flags", "a character following a path of waypoints is killed (what happens to the waypoints)". Then figure out a design that makes all this functionality the most straightforward and easiest to implement.



#16 Dominik2000   Members   -  Reputation: 249

Like
0Likes
Like

Posted 01 October 2013 - 12:13 AM


E.g. I don't know where to put my money system. This I would place in the logic. Or is this another system?
 
What do you envision your money system doing? It's hard to give an opinion without knowing what functionality you require.

 

In my game I want to set a building on an user defined place, that means he can choose, what to build on a gui, and then place it with the mouse cursor. So the money system has to check, has the user enough money, to buy this building, and if yes, to substract the defined price.

 


Dominik2000, on 30 Sept 2013 - 3:19 PM, said:

Yes I don't have drawn the Input system, sorry for this. And how should I do this without event messaging? How could I exit the main loop?
 
I'm sorry, I'm not really sure what you're asking here. During your update tick, your InputSystem should be able to query what key/mouse/gamepad events have taken place since the last tick.

 

My fault, dumb question.

 


Dominik2000, on 30 Sept 2013 - 3:19 PM, said:

Yes but I want to have one place to store all waypoints and pathes to other waypoints. In my opinion every entity has it's own waypoint component. So I have much duplicate data in store...
 
You really should avoid any duplicate data. Without knowing more about how your waypoints are created/used/modified (are they static points on a map? Is the player interacting with the game to create them? Is the AI using them to move game objects along a path?), it's hard to offer advice. You could certainly make your way-points completely separate from your entity-component-system architecture - if that makes sense. But if you find, say, that you're suddenly writing a ton of code to synchronize the game entities with your waypoint manager, then maybe it makes more sense to have them tightly integrated with your ECS.
 
Try listing out the various scenarios involving waypoints. For instance: "the user adds a waypoint to his character's path with the mouse", "some way-points are drawn as floating white flags", "a character following a path of waypoints is killed (what happens to the waypoints)". Then figure out a design that makes all this functionality the most straightforward and easiest to implement.

 

Waypoints will be set on placing buildings in the game (buildings can have waypoints, the user can place also streets). I haven't exactly figured out, how to save this data best. I think that I should have one data store where all the waypoints and pathes to other waypoints are stored, and the entity has a component where the waypoints, it has to drive along, are stored. This global waypoint store needs functionality to calc the best way to one defined waypoint. Maybe it's good to have also a waypoint manager where to store and a waypoint system with components to calculate the way.

 

Is this enough information? 



#17 phil_t   Crossbones+   -  Reputation: 3234

Like
1Likes
Like

Posted 01 October 2013 - 03:21 PM


In my game I want to set a building on an user defined place, that means he can choose, what to build on a gui, and then place it with the mouse cursor. So the money system has to check, has the user enough money, to buy this building, and if yes, to substract the defined price.

 

Checking a value and subtracting it seems like a fairly trivial operation - so if that's all it is, I don't think it merits a system. Taking a stab at it: I would think that there would be player entity that has a "PlayerInfo" component which has a Money property (among other things that are specific to player entities). The code that runs the GUI for choosing buildings would operate in part off of the PlayerInfo component. That code can just check the Money value to see which buildings are available for the player to build, etc..

 

Note that by "player entity", I don't mean the objects that are owned by players, but an entity that represents the player overall. For instance, if there were 2 players: human vs AI, then there would be two of these entities.

 


Waypoints will be set on placing buildings in the game (buildings can have waypoints, the user can place also streets). I haven't exactly figured out, how to save this data best. I think that I should have one data store where all the waypoints and pathes to other waypoints are stored, and the entity has a component where the waypoints, it has to drive along, are stored. This global waypoint store needs functionality to calc the best way to one defined waypoint. Maybe it's good to have also a waypoint manager where to store and a waypoint system with components to calculate the way.

 

Well I can't really suggest anything concrete from that. It sounds like a fairly complicated setup. If you are having trouble coming up with a design, you might want to try prototyping something to see where "gotchas" come up.



#18 Reitano   Members   -  Reputation: 488

Like
1Likes
Like

Posted 02 October 2013 - 04:59 AM

My advice is that of decomposing the engine in libraries designed with the principles of middlewares: fully decoupled, configurable, robust, reusable, testable. Then, build an entity/component framework at a higher level with a common API for creating, destroying, updating and synchronizing components of any type.

 

With that in mind, you would first develop an AI library with support for steering behaviours, waypoints, pathfinding etc. Have a look at commercial or open source AI libraries for inspiration. Among other things, the AI library is responsible for allocating, destroying, loading, saving and updating AI primitives. AI data and code, being fully decoupled, are optimal and do not suffer from the usual pitfalls of overly generic code.

Then, as part of the entity framework, you would have one component system per AI primitive: an AIAgentComponentSystem, an AIWaypointComponentSystem etc. These systems internally communicate with the AI library. They also synchronize AI components with other components like the TransformationComponent; coupling is at the entity level, not the library level.

 

In your case, an entity moving around the scene following a path could be configured as follows:

<entity name="player">
  <component class="Transformation"/>
  <component class="AIAgent"/>
    <wayPoint>ID</wayPoint>
  </component>
  <component class="Mesh"/>
  <component class="Script"/>
</entity>

The AIAgent component internally owns an AI Agent managed by the AI library. It syncs the agent position with the Transformation component which is then syncronized with the Mesh component for visualization. The Script component contains game logic and allows you to configure the pathfinding at run-time based on game events (i.e: switch pathfinding on/off or pass control to physics).

Then, if desirable, you would have an entity per AI waypoint:

<entity name="waypoint 0">
  <component class="AIWayPoint">
    <ID>423432423</ID>
    <file>waypoint0.dat</file>
  </component>
</entity>

Note that waypoints do not necessarily have to have an associated component if you do not need to manipulate them with the component API.

Please let me know if you need further clarifications as I've written this in a rush.



#19 Dominik2000   Members   -  Reputation: 249

Like
0Likes
Like

Posted 03 October 2013 - 03:46 AM

Hmm, this is also a interesting idea.

 

A little bit more clarification is needed. In my diagram the systems will be removed and the entity manager has only components? The AI libary will offer a pointer to it's managed AI Agent and this will be stored in the component?

 

It's not completly clear to me.

 

Dominik



#20 Reitano   Members   -  Reputation: 488

Like
0Likes
Like

Posted 07 October 2013 - 05:48 AM

 

A little bit more clarification is needed. In my diagram the systems will be removed and the entity manager has only components? The AI libary will offer a pointer to it's managed AI Agent and this will be stored in the component?

Hi Dominik, let me show you the organization of my Visual Studio solution in this case:

Projects:
- AI library: re-usable, fully decoupled and configurable, could be plugged in other engines
- Entity/component framework: the framework for the management of scenes, entities and component systems.
- AI module: an engine module which, among other tasks, initializes the AI library and adds AI component systems to the entity framework. The AI library is injected as a dependency into them.

The AI library allocates and returns instances of AI agents. In my component framework, there is no Component class (one of my key requirements) and everything can be a component so the AI agent pointer is managed directly. If you have a Component class, then you'll need an AIAgentComponent proxy that stores the AI agent pointer and dispatches calls to it.

 

Please let me know if you have other questions.






Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS