I'm working on upgrading my component based entity system. Right now I have a hierarchy of Entity objects within a Scene (which is itself just a special Entity). Each Entity has components that implement a Component interface that are basically a collection of callbacks. Components have onAttach, onUpdate, onDetach, etc. callback functions that are called when these events happen. The problem here though, is that I've got a really random memory access pattern by traversing through the hierarchy every frame and calling into each component. What I would like is to have a more linear memory access pattern where all components of the same type are updated together all at once.

I tried making it so my Component classes were just structures of data, and have a System that actually implemented the functionality. The problem with that though is actually tracking components. Should I require that when an Entity is created it will have a pointer to some sort of "SystemManager" that it can notify when Components are attached or detached? And if I do that, then how would I be able to retain support for my features like disabling entities and all their children? I can't just check the owner Entity of the Component to see if it's disabled, because that would throw another random memory access back in, and it would create more work when I have to traverse the hierarchy upward until the root to see if any parents are disabled.

Basically, all I want to do is be able to make my memory accesses more linear to improve cache performance while retaining my features (mainly creating a Scene in memory without setting it as active; and being able to disable an object and all its children). I don't care if I have to separate functionality from components, in fact I'd prefer to do so because of the extra opportunities for parallelism it presents in the future.

Should I require that when an Entity is created it will have a pointer to some sort of "SystemManager" that it can notify when Components are attached or detached?

Is there a reason the entity needs to be in the picture when adding/removing components? If someone else is managing the component storage, then no "notifications" are needed. Instead of "hey entity, I'm adding a foo component to you", have it be "hey worldmanagerthing, I'm adding a foo component to entity 12345".

And if I do that, then how would I be able to retain support for my features like disabling entities and all their children?

Do you have functionality for deleting an entity and all its children? Maybe this could be similar, except that instead of the entity/components being reclaimed, they could be moved to some alternate storage area while disabled.

The main reason the Entity is in the picture when adding components is so that I can access an Component directly with an index to it in the Entity. I guess if I used some sort of lookup table for entities and components in the scene then it wouldn't be too bad, although that would be some extra searching to add to it.

Yeah I've got a removeAllChildren method, and I've got a regular remove method that also cleans up the children. Basically what you're suggesting is that I recurse through the children when an entity is disabled and remove their components from the manager right?

In a true ECS, you don't need the Entity to access a component. The Entity is just an ID. The component is managed by some kind of System. If you want to query the component on the Entity, you just ask the corresponding System if it has a component for the given ID.

For removal, you can have the systems register a callback for when any Entity is removed/disabled which receives the ID, and then checks if they have the component and removes it if so. Assuming you don't have hundreds of different component, this won't be a real perf problem in practical uses.

See BitSquid's blog for more ideas on how to implement all this. Their whole blog is pretty good, but the article series starting on that link is relevant to this post.

--

Personally, I think ECS are an overhyped game hobbyist fad. Big AAA games have perfectly good performance with simpler and more manageable component models (shocker: many don't even have components at all). Simpler component models are much easier to work with and make it far easier to add new components or maintain game code with less bugs and less random guessing about performance problems you won't actually have anyway. You can trivially solve cache access pattern problems without contorting yourself into an ECS and you can highly parallelize your modules without even touching your game objects... assuming you kept things simple and didn't over-engineer a ton of nonsense.

The whole ECS thing evolved in ways few seem to understand. Scott Bilas' original paper on componentized game engines from Gas Powered Games's Dungeon Siege touched lightly on component allocation and compact arrays of component pools, but DS did not in any way use what you think of as an ECS today.

The blog that popularized the term ECS and revitalized interest in it over simpler component models was written by a Java guy. Java is a rather bad language for games and you _have_ to do the micro-aspect ECS-like component model in Java to get a decent setup. In Java, (a) you can't control memory layout or allocation strategy of class types, (b) Java didn't and still doesn't have proper parameterized algorithm and data structure support, and (c) no RAII or smart handles. These two limitations required you to stick to arrays of primitive data types and C-like procedural-ish code in order to get decent performance for large numbers of game objects. C++ doesn't have these limitations and you can write good OOP code that also has top-notch performance.

You can be both data-oriented _and_ object-oriented. They're design principles, not strict ideologies.

</off-topic-rant>

Just thought I'd share what I eventually decided on in case anyone else was interested. The responses I got were really helpful and made me start thinking about my component system in a different way.

The way it's setup now, is I run a quick recursion through the tree of objects each frame, and add their components to update lists for that frame. I pay the cost of recursing through the entire scene hierarchy, but I still retained my hierarchies, enabling/disabling still works correctly, and my Entity/Scene management code has become simpler. I figure that I'll probably only be recursing 5 levels deep at max, because the simple objects making up the majority of most scenes are probably going to be self contained static meshes with no hierarchy anyways. I also handle initialization with flags on the components. Each component has flags, and one of the flags is just a "has been initialized" flag that gets set after initializing that component.

Anyways, I just figured I'd post what I ended up going with in case anyone found it interesting.

out of curiosity, why the tree of entities? what types of entities have child entities? its seems a bit unusual:

"the simple objects making up the majority of most scenes are probably going to be self contained static meshes with no hierarchy anyways."

have you actually identified objects in the game that have hierarchies? or is this just a capability you anticipate needing?

so what type of objects have a hierarchy? i've heard of using tree hierarchies for renderables: "horse - rider - sword" type thing, but i've never heard of it used for entities.

out of curiosity, why the tree of entities? what types of entities have child entities? its seems a bit unusual:

"the simple objects making up the majority of most scenes are probably going to be self contained static meshes with no hierarchy anyways."

have you actually identified objects in the game that have hierarchies? or is this just a capability you anticipate needing?

so what type of objects have a hierarchy? i've heard of using tree hierarchies for renderables: "horse - rider - sword" type thing, but i've never heard of it used for entities.

It's more convenient for gameplay code. If someone gets in a vehicle I can set their character to be parented to the vehicle and move everything with it automatically that way. I can attach weapon entities to bones, and I can keep the hitboxes attached to the bones easily too. Unity does it the same way. You've got a game object and a bunch of components and children attached to it. Also, by making each bone in a skeleton a game object I can write my animation system to work for all game objects instead of just skeletons.

Though if there was a clean way to retain that functionality and switch to a plain array of objects then I'd probably at least try it out since it would simplify things quite a bit. I guess I could implement the hierarchy as a component, or implement a scene graph as part of the rendering system instead... But it just seems like I would lose quite a bit of flexibility when hitboxes need to match up with animations.

out of curiosity, why the tree of entities? what types of entities have child entities? its seems a bit unusual:

"the simple objects making up the majority of most scenes are probably going to be self contained static meshes with no hierarchy anyways."

have you actually identified objects in the game that have hierarchies? or is this just a capability you anticipate needing?

so what type of objects have a hierarchy? i've heard of using tree hierarchies for renderables: "horse - rider - sword" type thing, but i've never heard of it used for entities.

It's more convenient for gameplay code. If someone gets in a vehicle I can set their character to be parented to the vehicle and move everything with it automatically that way. I can attach weapon entities to bones, and I can keep the hitboxes attached to the bones easily too. Unity does it the same way. You've got a game object and a bunch of components and children attached to it. Also, by making each bone in a skeleton a game object I can write my animation system to work for all game objects instead of just skeletons.

Though if there was a clean way to retain that functionality and switch to a plain array of objects then I'd probably at least try it out since it would simplify things quite a bit. I guess I could implement the hierarchy as a component, or implement a scene graph as part of the rendering system instead... But it just seems like I would lose quite a bit of flexibility when hitboxes need to match up with animations.

I'd really recommend against using entity parent/child relationships to represent skeletal attachments -- this is basically scene-graphs all over again.

There's other properties besides attachments that deserve parent/child relationships - e.g.
* if you place a mesh as a child of a material, should it use that material?
* if a material is a child of a camera, should it render the camera to a texture and then use that's it's colour?
* if a flame-area-of-effect is a child of a flame-thrower, should their lifetimes be bound together (should deleting the weapon delete the flame)?
* if a grenade is a child of a player, should it appear in their inventory?

Old school scene graphs used to do all that and more...

You can't choose to support everything here, because many are contradictory - if I put a cup on a table, and parent it to the table so they move together, then I don't want the cup to be deleted if the table is destroyed...

Personally, I'd reserve entity parenting only for lifetime management - the same as lifetime management in C++.

To implement a transform hierarchy, I'd have a transform component with a matrix and a parent-transform-id. That's an array of matrices and an array of indexes.
Entities with positions can have a transform component ID. Skeletons can have an array of transforms component IDs.

The root bone of a player can then set a vehicle's seat bone as it's parent, without the player entity actually being a child of the vehicle.

If you have other relationships between components, you can implement them the same way, instead of hijacking entity connections themselves.

That sounds like a pretty convenient way to do it... And it frees me from the transform hierarchy when I'm doing the 2D rendering for interface and HUD. I'll try a quick prototype and see what kind of results I get. Attaching weapons and hitboxes is the main concern I had, but using transform components to build the hierarchy should fix that.

Though if there was a clean way to retain that functionality and switch to a plain array of objects then I'd probably at least try it out since it would simplify things quite a bit.

one possibility:

use an array[max_object] of objects.

each object has a parent_ID.

each object has a num_childeren (where max_children = 10 or whatever)

each object has an array[max_children] of child_IDs.

parent ID lets you traverse up the hierarchy.

child ID lets you traverse down the hierarchy.

a parent id of -1 means no parent.

num_children=0 means no children.

pros: fast arrays, can be iterated quickly. can be a static data structure.

cons: limited to max_objects, max_children.

this is a flat array implementation of a tree, a variation on the flat array implementation of a doubly linked list (IE one that has multiple "next" pointers).

i like the idea of object hierarchies for movement. i only use that type of thing in one place - moving entities and dropped objects aboard moving rafts. i render equipment using hierarchies, but don't consider them objects, simply inventory items.

converting my non-hierarchy entity list to this style would be straightforward. add the parent and child variables, and modify update to additionally apply results to any offspring. when an entity boards or disembarks from a raft, set parent and child variables for the entity and raft. max_children could be a problem if they want to drop 100 objects on the deck! <g>.