I'm trying to implement a component based system for the first time and I understand the core concepts but I'm having trouble working out a framework that allows flexibility but is also cache friendly.

For example here's my Transform component:

struct TransformComponent
{
 glm::mat4x4* localTransform;
 glm::mat4x4* worldTransform;
 int* parent;
};

and Here's an abridged version of my transform system:

class TransformSystem
{
 vector<glm::mat4x4> worldTransforms;
 vector<glm::mat4x4> localTransforms;
 vector<int> parents;
 vector<size_t> denseToSpare;
 vector<InternalHandle> sparseToDense;
 
 size_t componentCount;
 size_t firstFree
 
 Handle createComponent();
 void destroyComponent(Handle target);
 TransformComponent getComponent(Handle target);
 void setParent(Handle child, Handle parent);
 void update()
 {
   glm::mat4x4* wT = &worldTransforms[0];
   glm::mat4x4* lT = &localTransforms[0];
   int* parent = &parents[0];
   for(size_t i = 0; i < componentCount; i++, wT++, lT++, parent++)
   { 
     if(parent == -1)
        wT = lT;
     else
       wT = worldTransforms[parent] * lT;
   }
 }
}

Here's the entity structure

struct  Entity
{
   unsigned int ID;
   hash_map<ComponentType, Handle> componentHandles;
};

-Components are stored in a dense array which is managed by a sparse array of handles.

-External "Handle"s contain an index into the sparse array and a version number

-Internal "InternalHandle"s contain an index into the dense array, a version number and an index for keeping a free list of internal handles in the sparse array

This all works fine. The trouble is inter-component and inter-system communication. For example the MeshRendererComponent, and associated system, are dependent upon the TransformComponent for position.

I've spent hours looking at this problem from every angle I can think of and I can't just seem to figure out a reasonable solution to retrieving the component from the other system without making the components aware of their entity.

If anybody has any insights they would like to share I would be very grateful. Thank you.

Is think thats usually done using some kind of bitmask on the entity, lets say you have 3 components

1 position

2 mesh

3 movement

One entity has all three components giving 111 and your MovementSystem which needs position and movement needs 101, so it would get this Entity

Thanks for the responses. So to clear some things up.

It's unclear from his post where things are actually stored. All that's in the Entity itself are handles for each component type, not the components themselves.

Transforms seem to be stored in the TransformSystem - or at least the data that contains them appears to be (the vectors of world/local matrices and parent indices). And TransformComponent has pointers into that array? Is there a reason you implemented it this way?

Currently the transforms are stored inside the transform system. It may make more sense to store them in some kind of component pool outisde of the systems.

The parent component is an index into the dense array. The theory being that as long as the parent appears in the array prior to the child (p < i) then the transforms can be calculated in a single (cache-friendly) linear pass rather than having to dance around a tree structure which could be anywhere in memory.

I would instead just have the two matrices and parent be directly part of TransformComponent. Instead of "many arrays of one", you should instead make "one array of many".

I got carried away with the "structure of arrays" concept. In hindsight there's very little point in separating them like that as every component of the structure is accessed in every instance. So I will combine them into a structure.

I think what I'll do is move all of the components out of the systems and in to a component manager where each pool can be accessed by some kind of type identifier.

Some code off the top of my head

Note: my VBO's are assigned at the max size indexable by an unsigned short and then broken into blocks by a page/pool allocator and I use managers to track the currently bound vao, ibo, vbo, shader etc to minimise state change.

struct MeshRendererComponent
{
  VertexBufferBlock* VertexBuffer;
  ShaderProgram* Shader;
  Handle TransformHandle;
};
class MeshRendererSystem
{
   update()
   {
      MeshRendererComponent* currentComponent = ComponentManager.getPool<MeshRenderComponent>().front();
      ComponentPool<TransformComponent>* transformPool = &componentManager.getPool<TransformComponent>();
      for(size_t i = 0; i < numComponents; i++, currentComponent++)
      {
         ShaderManager::Bind(currentComponent->Shader);
         
         //Ignore this. I'm not done with the shader system yet
         currentComponent->Shader->setMatrix4x4("modelMatrix", transformPool[currentComponent->TransformHandle]);
 
         VertexBufferManager::DrawBlock(currentComponent->VertexBuffer);
      }
   }
};

I attempted this in one of my revisions of my entity/component framework and I am curious how you approached a few concerns I found with it.

My initial thought was to have ComponentPool<T> have a constructor argument that controlled the maximum number of components the pool could maintain. This inevitability acted much like calling reserve(size) on a vector, eliminating any future allocations mid-frame that could impact the main loop's execution. I saw this as a great way to have total control over the memory budget of varying systems and this upper-cap could either be some static cap or data-driven by scene metadata.

One of my first concerns I saw with it was how to handle component releasing:

ComponentPool<BoxCollider>* pColliderPool = GetComponentManager().GetPool<BoxCollider>();
BoxCollider& collider1 = pColliderPool->GetComponent(colliderId1); //! get box collider with id colliderId1
pColliderPool->Release(colliderId2); //! free collider with id colliderId2
collider1.someProperty = someValue;

When a component is released, it seems pretty common that the version aspect of the component id/handle gets incremented; thus invalidating any existing external references if they attempt to use the handle/id in future operations. Additionally, releasing a component also generally means that the dense array needs to be repacked by swapping the component at the freed index with the last active index.

The problem when you swap the two indices is that it invalidates any existing references up to that point and it invalidates iterators, much like what you'd expect of typical vectors. To address this, I thought about actually eliminating the swap step in the release process and impose some logic on the exposed iterator api to be capable of determining whether a pool index is occupied by an active element or one that has been released and simply skip the released ones.

But at some point, the dense array needs to be reorganized and actually be dense rather than have gaps. But this process needs to happen at secure points in the code to avoid problems, thus I exposed a Pack() method that essentially takes the released indices, swaps them with the last active element, updates the internal structure index references, etc.

What I am curious is whether this is somewhat common place and normal?

Are there other alternatives that I might not have considered to handling the release of a component?

What about drawbacks to this approach?

This approach does seem quite spot on to me, at least on the surface.

We've satisfied the need to keep components in contiguous memory for cache locality. Simple systems that need to iterate components can do so just by iteracting with the component pools directly. Systems that require additional information beyond what components expose can create their own internal structures, store them in cache friendly means and have those structures reference the pool's component id, allowing for one level of indirection lookup for a component's data where needed.

Where to draw the border between SoA or AoS (or in fact a combination of both) depends on the use case. Quoting the entity placement once more:

1.) For so-called static geometry no transform is needed because the vertex positions are already given in world co-ordinates. These entities are renderable but provide no placement component. This may be true for some special things like terrain, too.

2.) All other geometry has a placement component but not necessarily a dynamic one. This placement is obviously most meaningful in world co-ordinates.

3.) Dynamic placement may be done in world co-ordinates or in model co-ordinates, or perhaps in another local system. The dynamic may be driven by an animation or a mechanic (from which forward kinematic is one possibility).

If you now think of the amounts of occurrences you may notice that in general from a performance point of view (where doing nothing is usually the best choice) an AoS is not necessarily good.

Let's assume that there is a sub-system where WorldPlacement components are stored, and these are (well) given in world co-ordinates. Entities belonging to 1. will not be represented therein, so the array is lesser in its length than the renderable entity list is. Sound and graphic rendering can assume that the placements are up to date and ready to use when it comes to rendering at the end of the game loop. Geometry is tagged whether it has a placement or not (you can, if you will, also request for the WorldPlacement and assume that the geometry is in word co-ordinates if you don't get a placement back).

Let's further assume that there is a sub-system that stores LocalPlacement components, expressing a parent-child relationship with forward kinematic behavior. Each entry herein is defined to have such a relation, and it can be safely assumed that the length of the array is noticeable lesser than those of the WorldPlacement. If local placement chains are allowed, it is obviously best to have the LocalPlacement components in an order that guarantees that higher levels in a chain are already processed when a lower level is next.

Let's further assume that there is a sub-system that drives key framed animation. One typical use case is to do a Placement transform with it. It may effect a placement stored in WorldTransform or in LocalTransform. This is not necessarily a problem because animations are manifold and often switched at runtime, so doing some binding is probably needed anyway. And because from this point of view a placement is a placement regardless whether it is store in the WorldPlacement or else in the LocalPlacement sub-system.

With these assumptions, the first update runs on the animation sub-system. It does its job and alters some world and some local placements. The next update runs on the LocalPlacement sub-system (which is naturally more constrained than an animation). After this point all WorldPlacement components are up to date.

As can be seen, rendering, placement in the world, local placement, and mechanisms to alter placements in general not only can be separated but should be separated because they occur in different frequencies and at different update times. The only match here is that LocalPlacement can combine storage and transform in a simple way. Notice that the WorldPlacement sub-system is passive by itself. It stores the placements and provides a central access point, but it never changes them.