• Advertisement

Entity-Component systems

Recommended Posts

I read some interesting literature describing Entity-Component systems:

GREGORY J.: Game Engine Architecture, 2nd Edition.

http://www.randygaul.net/2013/05/20/component-based-engine-design/

http://www.randygaul.net/2014/06/10/sane-usage-of-components-and-entity-systems/

and some GameDev links (which I removed due to the ugly and large widgets replacing the urls).

Though, I am still not 100% sure, the direction I want to go. I like Unity's representation in the editor, unfortunately Unity is close source. Though, Unreal's monolithic class hierarchy is nice as well since you can basically override every class as a user.

The Unity editor has a GameObject tab and a Component tab. The "GameObjects" represent the different kind of nodes in the scene graph. The "components" seem to be add-ons for the "GameObjects". Though, some overlap is possible since you can have a standalone Camera or an add-on Camera. At first this seems like a nice separation. But when looking to the hierarchies in their scripting API, most of these "GameObjects" and "components" are just inheriting their Component class anyway.

If one wants to represent this in code, the GameObject can have:

  • human readable name (string or hashed string)
  • guid
  • some flags, tags and layers
  • transform (I do not want separate transform component)
  • collection of component (pointers or ids)
  • some message/event passing utilities

Each Component has a way to access its associated GameObject. My scripts will be components as well containing some specific message handling methods that can be overriden.

Each System will access the Components it needs.

That said, how are these Components stored at the Engine side? For memory coherence, I would expect to see something like a dynamic array per Component base type instead of per Component type since you can have an infinite number of components. I guess this Component base type must be specialized enough to avoid all types of casts?

Edited by matt77hias

Share this post


Link to post
Share on other sites
Advertisement

Unity is actually reworking it's EC System right now.

You might want to have a look at this. Also, the code is open source. (Or it's going to be soon) :

 

Share this post


Link to post
Share on other sites
3 hours ago, LifeIsGood said:

Unity is actually reworking it's EC System right now.

Interesting refactor, though for now I don't want to go to a complete SoA instead of AoS. That is a bit too data driven (which is problematic for encapsulation).

 

I am not specifically concerned with scripting alone. I want to put Components used by other Systems (such as audio, physics and graphics) in some collections as well. I probably will store these data by value. Though, what is the granularity of storing these values (i.e. how many collections)?

Furthermore, assume we have an infinite number of scripts (each with their own type extending a base class), how could you implement the equivalent of Unity's GetComponent method: T *Component::GetComponent< T > without using casts? How would you even now how to cast? The GameObject only has some pointers to abstract Components. If you want all SpotLights, you can simply look in your scene's SpotLightComponent buffer to find some potential matches. But what if you want a specific ScriptComponent?

Edited by matt77hias

Share this post


Link to post
Share on other sites
4 hours ago, matt77hias said:

Interesting refactor, though for now I don't want to go to a complete SoA instead of AoS. That is a bit too data driven (which is problematic for encapsulation).

 

I am not specifically concerned with scripting alone. I want to put Components used by other Systems (such as audio, physics and graphics) in some collections as well. I probably will store these data by value. Though, what is the granularity of storing these values (i.e. how many collections)?

Furthermore, assume we have an infinite number of scripts (each with their own type extending a base class), how could you implement the equivalent of Unity's GetComponent method: T *Component::GetComponent< T > without using casts? How would you even now how to cast? The GameObject only has some pointers to abstract Components. If you want all SpotLights, you can simply look in your scene's SpotLightComponent buffer to find some potential matches. But what if you want a specific ScriptComponent?

If you don't mind me asking, why are you choosing the ecs model over entity-component model if you're not going to take certain optimizations like DoD and etc? You will be creating a more complicated system without the benefits of cache coherency. Unity and UE4 are currently using an entity-component model. A benefit is that it's much simpler to write code and encapsulate, but it becomes more difficult to multithread it. To deal with the issue of multithreading, Unity has chosen to switch to an ecs model. I'm currently maintaining the entity-component model, but using messaging to handle multithreading.

Share this post


Link to post
Share on other sites
10 hours ago, AxeGuywithanAxe said:

ecs model over entity-component model

What is the difference between an Entity-Component-System and an Entity-Component Model in your terminology? Nor my Entities or Components contain methods like do this or do that apart from the scripts? My rendering system retrieves the data from the components and performs the rendering based on that data; so quite DoD.

I am still doubting whether a SpotLight is the most specialized component? Assume it is, we can store SpotLights by value in some collection. If not, the user can extend the behavior (within the realm of engine support) via inheritance.

Share this post


Link to post
Share on other sites
52 minutes ago, Hodgman said:

ECS means you're using tables of data like in the relational model. 

EC means your using composition like in OOD. 

Than I rather like the terminology of Game Engine Architecture: The Object Model architecture can be object-centric of property-centric (i.e. data tables).

Apparently, the object-centric approach encapsulates attributes and behaviors. That latter is definitely not something that I want. My components are represented as stupid containers where the attributes are encapsulated but no behavior is added (except for scripts since their behavior is actually their data). The property-centric approach encapsulates only the attributes whereas the behavior is implicitly defined by the attributes.

On the other hand, what I actually intend to do falls under object-centric.

Generic Components (pp. 885-886): The root game object is provided with a generic list of components which all derive from a common base class (to retrieve type information and pass events/messages).

 

 

Edited by matt77hias

Share this post


Link to post
Share on other sites

I provide some pseudocode:

class Entity {
    
    // Identification
    U32 m_guid;
    string m_name;
    vector< string > m_tags;
    vector< string > m_layers;
    
    // State
    bool m_active;
    
    // Components
    UniquePtr< TransformNode > m_transform;
    vector< Component * > m_components; ~~~> Accessible from the scripts without casting???
}

class Component {

    // Owner
    Entity *m_entity;
    
    // State
    bool m_active;
}

class Scene {

    // Entities
    vector< Entity > m_entities; ~~~> All systems will access the TransformNode component

    // Component collections
    vector< AudioEmitter > m_audioemitters; ~~~> AudioSystem
    ...
    vector< BoxCollider > m_boxcolliders; ~~~> PhysicsSystem
    ...
    vector< Spotlight > m_spotlights; ~~~> RenderingSystem
    ...
    vector< UniquePtr< Script > > m_scripts; ~~~> ScriptSystem (note the pointer)
}

Not really sure how frequently a user wants to remove Components from an Entity, so this is a potential bottleneck. Idd. for searching a specific Component (e.g. via a script).

 

Furthermore, assume I want to visualize the tree hierarchy in an editor, then I need to know the type of each Component to access its attributes or:

  • Each Component should have a virtual method DrawUI. But that makes no sense.
  • Each Component should have a message handler for visualization purposes. But that makes no sense either. Furthermore, this is conceptually pretty similar to the first solution.
  • Each Component should be accessible by a Visitor. The editor is then capable of using a visitor for showing the UI.
Edited by matt77hias

Share this post


Link to post
Share on other sites

I think i'm starting to understand what you are saying, but it seems like you've also noticed that you hit one of the problems of ECS . Which is leveraging the "system" part while still maintaining the benefits of certain practices like inheritance. 

 

On 12/2/2017 at 6:43 AM, matt77hias said:

That said, how are these Components stored at the Engine side? For memory coherence, I would expect to see something like a dynamic array per Component base type instead of per Component type since you can have an infinite number of components. I guess this Component base type must be specialized enough to avoid all types of casts?

The issue with this approach, is that it isn't exactly cache coherent. You are still jumping through memory to access the components, so in what way have you actually benefited? This and a few other issues , for instance not having an easy way to update all of a parent entity's components prior to my own, is why I chose to go with the entity component model in which components contain logic. 

Share this post


Link to post
Share on other sites
53 minutes ago, AxeGuywithanAxe said:

The issue with this approach, is that it isn't exactly cache coherent.

For the scripts, I indeed use pointers to exploit polymorphism. The user is capable of creating an infinite number of different scripts. For the other components, it is possible to store them by value to exploit cache coherence. The user is not capable of creating such components: they are defined by the engine itself

 

56 minutes ago, AxeGuywithanAxe said:

for instance not having an easy way to update all of a parent entity's components prior to my own

.My TransformNode constitutes the scene graph. Though, you are indeed right that if you use some information not contained in the local part of the TransformNode (for example something relative to the world), the correctness depend on the order of executing the scripts.

Game Engine Architecture recommends using buckets, but this is way from general (Naughty Dog uses only 3 buckets).

1 hour ago, AxeGuywithanAxe said:

The issue with this approach, is that it isn't exactly cache coherent. You are still jumping through memory to access the components, so in what way have you actually benefited? This and a few other issues , for instance not having an easy way to update all of a parent entity's components prior to my own, is why I chose to go with the entity component model in which components contain logic. 

So based on the above, is it correct that you are only concerned with:

vector< UniquePtr< Script > > m_scripts; ~~~> ScriptSystem (note the pointer)

because apart from the scripts, all my other components' "behavior" is to get, set or bind some data on behalf of some 3th party.

 

1 hour ago, AxeGuywithanAxe said:

is why I chose to go with the entity component model in which components contain logic

But how does this look like from the perspective of the Entity? One common Component base class (OO way of doing things) or a "sort of database" (property/table way of doing things)?

Share this post


Link to post
Share on other sites
1 hour ago, matt77hias said:

For the scripts, I indeed use pointers to exploit polymorphism. The user is capable of creating an infinite number of different scripts. For the other components, it is possible to store them by value to exploit cache coherence. The user is not capable of creating such components: they are defined by the engine itself

Well if you're going to store them by value, then you are effectively doing what is being done in the Unity video, just slightly less efficient. I have yet to personally find a game that requires such an approach that the benefits outway the cost of increased complexity, but it is a valid approach.

 

1 hour ago, matt77hias said:

.My TransformNode constitutes the scene graph. Though, you are indeed right that if you use some information not contained in the local part of the TransformNode (for example something relative to the world), the correctness depend on the order of executing the scripts.

Yeah, I didn't take the bucketed approach because it has several problems, especially when you decide you want to multithread your engine. Any time spent syncing jobs is time wasted, and a bucketed approach is full of such issues.

 

1 hour ago, matt77hias said:

So based on the above, is it correct that you are only concerned with

From a "cache coherency" perspective , then the answer is yes.

 

1 hour ago, matt77hias said:

But how does this look like from the perspective of the Entity? One common Component base class (OO way of doing things) or a "sort of database" (property/table way of doing things)?



class CEntity
{

	Array<CComponent*>	m_Components;
};
      
// base for all components
class CComponent
{
      virtual void ActivateComponent();
      virtual void DeactivateComponent();
      virtual void TickComponent(float DeltaTime);     
};
      
class CPhysicsComponent : public CComponent
{
      
      
};
      
class CVisualComponent : public CComponent
{
      
      
};

There are many approaches to the EC model, but this is the basics of it.

Share this post


Link to post
Share on other sites
1 hour ago, AxeGuywithanAxe said:

virtual void TickComponent(float DeltaTime);

Isn't this behavior?

 

All methods are virtual? If I am correct the first two are just specific message handlers. But I do not get the Tick? This involves encapsulating the behavior in the component?

 

1 hour ago, AxeGuywithanAxe said:

then you are effectively doing what is being done in the Unity video

But Unity uses by value for scripts as well. With the "other components", I mean all non-script/MonoBehavior related components such as Models, Lights, Colliders, etc. Based on the video, I have no idea, how Unity is storing these on the C++ engine side.

1 hour ago, AxeGuywithanAxe said:

just slightly less efficient. I have yet to personally find a game that requires such an approach that the benefits outway the cost of increased complexity, but it is a valid approach

Could you elaborate?

Edited by matt77hias

Share this post


Link to post
Share on other sites
1 hour ago, AxeGuywithanAxe said:

There are many approaches to the EC model, but this is the basics of it.

And assume an instance of CEntity has a pointer to CPhysicsComponent. Is there a clean way to get that component, given the entity?

Share this post


Link to post
Share on other sites
37 minutes ago, matt77hias said:

Isn't this behavior?

 

All methods are virtual? If I am correct the first two are just specific message handlers. But I do not get the Tick? This involves encapsulating the behavior in the component?

Yeah, the entity-component model has behavior in the components. It takes the idea of composition , so instead of having a hiearchy like

CEntity -> CPhysicalEntity -> CMovableEntity , you'll have an entity, and you'll add a physicscomponent and a movecomponent to it. Not all functions have to be virtual, I was just giving an example. It's the same approach you would with any OOP hiearchy, you just separate "logic" into component types, i.e physics, visual , movement, script, etc. In my game engine, different component types have speciific logic when they are activated and deactivated, so they are in the base class, and most components have some sort of "tick" logic.

 

39 minutes ago, matt77hias said:

But Unity uses by value for scripts as well. With the "other components", I mean all non-script/MonoBehavior related components such as Models, Lights, Colliders, etc. Based on the video, I have no idea, how Unity is storing these on the C++ engine side.

You'll have to be more descriptive of your method, in most ecs approaches, scripts are handled by using a "CScriptComponent", so you would have to explain what your scripts actually do, and the point of them in more depth.

 

40 minutes ago, matt77hias said:

Could you elaborate?

Well with an ECS method, you will have data in one class and logic in the system class. This means that you're adding two classes in an entity - component -system model for every one class you would have made using an entity component model. I say I've yet to see one that's worth the added complexity because the largest games that exist , the Witcher 3 , Horizon, UE4 based games , Tomb Raider and etc all use an entity-component model. The only two major game engines that I know of that use ECS are the Overwatch engine , and the Insomniac engine (which isn't pure ECS because logic is still maintained in the components as far as I can tell from presentations).

40 minutes ago, matt77hias said:

And assume an instance of CEntity has a pointer to CPhysicsComponent. Is there a clean way to get that component, given the entity?

Yeah , I have a function called "GetComponentOfClass" and "GetComponentsOfClass". I programmed an automatic reflection system that preprocesses my c++ code and generates reflection data for classes, enums, structs, functions, properties, etc, but there are dozens of simple ways to create custom type info.

Share this post


Link to post
Share on other sites
12 hours ago, matt77hias said:

My rendering system retrieves the data from the components and performs the rendering based on that data; so quite DoD.

 

6 hours ago, matt77hias said:

UniquePtr< TransformNode > m_transform;

This really doesn't look like you're using any DoD. From the look of the code, it seems like you'd have something like this?

for each render component as renderable
  transform = renderable->m_entity->m_transform
  set_draw_data( transform )
  draw( renderable )

 

In DoD, you'd consider two things first: What function consumes transform data (e.g. the renderer), and what function(s) produce transform data (e.g. gameplay, animation, etc). Then consider in isolation, how the consumer function would ideally lay out the input data so that it can process it as quickly as possible. Then consider in isolation how the producer functions would ideally lay out their data so that they can produce it as quickly as possible. Then reconcile those two ideal layouts into something that works in practice. Figure out if you need any conversion transforms in the middle (e.g. gameplay outputs data set X, which is then converted to data set Y, which is consumed by the renderer).

12 hours ago, matt77hias said:

If not, the user can extend the behavior (within the realm of engine support) via inheritance.

Inheritance is not a tool for code re-use / extension. Composition is a tool for code re-use / extension.

Share this post


Link to post
Share on other sites
12 hours ago, AxeGuywithanAxe said:

In my game engine, different component types have speciific logic when they are activated and deactivated, so they are in the base class, and most components have some sort of "tick" logic.

But that is something, I wanted to avoid. In the beginning a Model was capable of drawing itself by drawing its Mesh and Material. If we avoid inheritance, we can make all these methods non-virtual to gain some performance, but this looks not very DoD, but rather OoD? So currently, I really extract the data from the scene and render the complete batch of Models based on the data of the Model, but not inside the Model. Why would a Model anyway be capable of drawing itself? It merely provides a description about its appearance (declarative vs imperative)?

12 hours ago, AxeGuywithanAxe said:

You'll have to be more descriptive of your method, in most ecs approaches, scripts are handled by using a "CScriptComponent", so you would have to explain what your scripts actually do, and the point of them in more depth.

My scripts do nearly everything at the moment, similar to Unity's MonoBehavior. So there is a duality, I do not like about my codebase:

  • Models contain a description of their appearance, but contain no drawing logic. A render pass will take care of that based on the data.
  • Scripts contain some typical hooks Update etc. in which they are capable of executing some operations (so not very descriptive at all). Currently, this basically involves changing some parameters of whatever at runtime (materials, transforms, etc.).
12 hours ago, AxeGuywithanAxe said:

Well with an ECS method, you will have data in one class and logic in the system class. This means that you're adding two classes in an entity - component -system model for every one class you would have made using an entity component model

Thanks this is a very clear definition. Though given this definition, I would prefer ECS over EC. Why would you add logic to the Component, because this way you will deviate from DoD to OoD? In the most extreme case, you can say that each E and C corresponds to a table in some relational database, which only contains data (so no logic)? ECS seems also somewhat similar to a producer-consumer system.

12 hours ago, AxeGuywithanAxe said:

Yeah , I have a function called "GetComponentOfClass" and "GetComponentsOfClass". I programmed an automatic reflection system that preprocesses my c++ code and generates reflection data for classes, enums, structs, functions, properties, etc, but there are dozens of simple ways to create custom type info.

And I guess, you use that as well for displaying each Component in the editor. Not only the engine defined ones, but user created scripts as well since you can basically display all member variables in an appropriate way. A visitor would be "clean" alternative for the engine defined ones, but could not anticipate the user defined ones without instructing the user to do so. So in that sense reflection is the most flexible solution. Though, you need to give up the data encapsulation provided by the typical getter/setter pair, since detecting an arbitrarily named getter/setter does not look very trivial (at least more difficult than making all member variables public).

12 hours ago, Hodgman said:

This really doesn't look like you're using any DoD. From the look of the code, it seems like you'd have something like this?


for each render component as renderable
  transform = renderable->m_entity->m_transform
  set_draw_data( transform )
  draw( renderable )

The renderer extracts the "renderables" which will be filtered in some collections (e.g. opaque, transparent, brdf, no brdf, etc.). The second line will be consequence of the ECS. At the moment I extend the entities, lacking components, but after refactoring it will indeed look something like that. The last two lines are sort of present as well. Am I misunderstanding something?

The following seems DoD to me from the perspective of the Components:

RenderSystem::set_draw_data(Transform)

RenderSystem::Draw(Renderable)

whereas this seems OoD to me from the perspective of the Components:

Transform::set_draw_data()

Renderable::Draw()

Edit: Though, I now also realize that DoD could mean packing all related data together independent of logical functionality. The drawback could be possible duplication.

12 hours ago, Hodgman said:

Then reconcile those two ideal layouts into something that works in practice. Figure out if you need any conversion transforms in the middle (e.g. gameplay outputs data set X, which is then converted to data set Y, which is consumed by the renderer).

But during the "conversion" you can have bad cache coherency. So isn't this moving the problem?

12 hours ago, Hodgman said:

Inheritance is not a tool for code re-use / extension. Composition is a tool for code re-use / extension.

UE4 uses this monolytic hierarchy, letting you extend everything. I agree that composition and aggregation are better practices, but inheritance sometimes look like the only way of hooking something to an existing system.

Edited by matt77hias

Share this post


Link to post
Share on other sites
3 hours ago, Hodgman said:

If I am going in this direction, I would never leave the design phase :o But I get the metaphor.

3 hours ago, Hodgman said:

What is the exact, minimal set of data required to fulfil the Transform::set_draw_data and Renderable::Draw functions.

One difficulty with this is: "what do you want to draw". I want to use a minimal number of GPU buffers so everything will be packed together: transform data, material data, etc. Though, some visualizations like the typical debug visualizations (solid, texture, etc.) require less data. But I understand that I should consider the most usual case (PBR) for that.

3 hours ago, Hodgman said:

outputting a list of indices into a data set in the first pass, and then collecting the data items associated with those indices in a second pass

But that will certainly result in jumping around in memory?

For instance, the Transform and the Drawable. Adding them together would be strange, since lots of different component handling systems require a Transform but a differnt Component. The rendering, physics, collision and audio system all need to be aware of a Transform and some specific Component. So, there will always be an indirection.

In this sense, transforms and scripts are the most difficult components imo. Transforms because everyone needs them and scripts because you can have an infinte number of types. I nearly get the impression that handling the latter efficiently is to let the user of a game engine write the scripting/gameplay system himself and hook it to the system.

3 hours ago, Hodgman said:

for each render component as renderable

But like I said for the renderables, i can store them by value in contiguous memory:

23 hours ago, matt77hias said:

vector< Spotlight > m_spotlights; ~~~> RenderingSystem

but it could be more efficient to collect the transforms as well and use one "conversion" structure while moving from the scene (producer) to the rendering system (consumer). But this requires going to the entity or in a pure component model going to the associated transform component (different locations in memory).

 

Currently I did the following:

void XM_CALLCONV VariableShadingPass::ProcessModels(
        const vector< const ModelNode * > &models,
        FXMMATRIX world_to_projection, 
        CXMMATRIX world_to_view, 
        CXMMATRIX view_to_world,
        bool transparency) {

        for (const auto node : models) {
            
            // Obtain node components (1/2).
            const TransformNode * const transform = node->GetTransform();
            const Model         * const model     = node->GetModel();
            const XMMATRIX object_to_world        = transform->GetObjectToWorldMatrix();
            const XMMATRIX object_to_projection   = object_to_world * world_to_projection;

            // Cull the model against the view frustum.
            if (ViewFrustum::Cull(object_to_projection, model->GetAABB())) {
                continue;
            }

            // Obtain node components (2/2).
            const XMMATRIX object_to_view         = object_to_world * world_to_view;
            const XMMATRIX world_to_object        = transform->GetWorldToObjectMatrix();
            const XMMATRIX view_to_object         = view_to_world * world_to_object;
            const XMMATRIX texture_transform      = node->GetTextureTransform()->GetTransformMatrix();
            const Material * const material       = model->GetMaterial();

            // Bind the model data.
            BindModelData(object_to_view, view_to_object, texture_transform, material);
            // Bind the pixel shader.
            BindPS(material, transparency);
            // Bind the model mesh.
            model->BindMesh(m_device_context);
            // Draw the model.
            model->Draw(m_device_context);
        }
    }

My taxonomy is:

  • Node
    • has a TransformNode
      • has a Transform
      • has a parent and childs
  • is derived by ModelNode
    • has a TextureTransform
    • has a Model
      • has a Mesh
      • has an offset and number of indices in the Mesh
      • has an immutable local AABB and BS
      • has a Material
  • is derived by OmniLightNode/SpotLightNode
    • has an OmniLight/SpotLight
  • is derived by OrthographicCameraNode/PerspectiveCameraNode
    • has a OrthographicCamera/PerspectiveCamera ~~> virtual function to get the view-to-projection and projection-to-view matrices
    • has viewport transform
  • ...

But I am going to remove the Node hierarchy, making all the remaining Components. Furthermore, I can store some data by value to avoid some pointer jumping.

 

 

Edited by matt77hias

Share this post


Link to post
Share on other sites
3 hours ago, Hodgman said:

A little OT, but this saved my day today, thanks for that awesome link!

Thats the thing every programmer should know and need to be aware of - most will call this "micro-optimizations" or "premature optimizations", but its just the correct way to think and code.

Now to get to the topic, that code:

void XM_CALLCONV VariableShadingPass::ProcessModels(
        const vector< const ModelNode * > &models,
        FXMMATRIX world_to_projection, 
        CXMMATRIX world_to_view, 
        CXMMATRIX view_to_world,
        bool transparency) {

        for (const auto node : models) {
            
            // Obtain node components (1/2).
            const TransformNode * const transform = node->GetTransform();
            const Model         * const model     = node->GetModel();
            const XMMATRIX object_to_world        = transform->GetObjectToWorldMatrix();
            const XMMATRIX object_to_projection   = object_to_world * world_to_projection;

            // Cull the model against the view frustum.
            if (ViewFrustum::Cull(object_to_projection, model->GetAABB())) {
                continue;
            }

            // Obtain node components (2/2).
            const XMMATRIX object_to_view         = object_to_world * world_to_view;
            const XMMATRIX world_to_object        = transform->GetWorldToObjectMatrix();
            const XMMATRIX view_to_object         = view_to_world * world_to_object;
            const XMMATRIX texture_transform      = node->GetTextureTransform()->GetTransformMatrix();
            const Material * const material       = model->GetMaterial();

            // Bind the model data.
            BindModelData(object_to_view, view_to_object, texture_transform, material);
            // Bind the pixel shader.
            BindPS(material, transparency);
            // Bind the model mesh.
            model->BindMesh(m_device_context);
            // Draw the model.
            model->Draw(m_device_context);
        }
    }

Does too many things:

- Compute matrices for culling

- Detect visbility for every model

- Compute matrices for rendering

- Draw the actual model

 

So to make this more cache friendly, make 3 functions:

- One which just computes all the matrices

- One which sort your models based on visibility

- One which draws the sorted models

Share this post


Link to post
Share on other sites
2 hours ago, Finalspace said:

So to make this more cache friendly, make 3 functions:

- One which just computes all the matrices

- One which sort your models based on visibility

- One which draws the sorted models

Thanks :)

You are absolutely right. It will look cleaner and smaller as well to use some helpers.

 

My largest concern, however, were the many pointer indirections.

 

I guess I will start with creating a reflection API first. Next, I will move my subentities to components and use some more by value storage. And, I will start refactoring all my render passes as well. Maybe, I can reduce the number as well.

Edited by matt77hias

Share this post


Link to post
Share on other sites

Does someone have some good references to reflection, especially introspection in C++17. The only thing C++17 adds that is helpful in this regard are structured bindings. So, I know that it is not going to work out of the box. I have seen some code depending on macros that need to be used for defining new structures or adapting existing ones, but that just doesn't look right. Alternatively, some APIs depend on accessing Clang's AST, but that is way to compiler specific.

 

Introspection can be handy for generating the UI of each component in an editor, but could you enforce constraints (the value must be in [0, 1]) and guard the correct setting of dirty bits (transformation matrices)?

Edited by matt77hias

Share this post


Link to post
Share on other sites

It honestly depends how in depth you want to go with reflection, in reality it can be extremely simple, to complex like code generation. Personally, I'd stick with a custom solution instead of using C++17, but that's just me.

This reflection system was started from Insomniac Games and expanded over time :

https://github.com/HeliumProject/Reflect

A little self promotion , this is a small blog about my old reflection system, it's had a major refactor and looks quite literally nothing like this now, but it worked at the time

http://www.ademolathompson.com/archetypesystem/

C++ Reflection is quite a large topic, so there are dozens of papers , blogs, etc about this exact topic if you google it.

A basic C++ implementation could be like this



class CClass
{
	typedef CObject*(*ClassConstructorType);

	CClass(const char* Name, .... ClassConstructorType Ref):
	...

	CObject* ConstructInstance() const 
	{
		return (*m_ClassConstructorRef)();
	}

private:

	// member parameters like size, alignment, etc
	uint32 	ClassSize;
	uint32  ClassAlignment;
	ClassConstructorType m_ClassConstructorRef;
	std::vector<CProperty*>	Properties;
    std::vector<CFunction*> Functions;
	std::string	ClassName;
    CClass* m_SuperClass;
};
      
// base class for all properties (int...float...etc)
      
class CProperty
{
      // functions to construct the value type, destruct the value type, copy, etc
 };
      

 // base class for all "reflected types"
      
 class CObject
 {
    virtual CClass* GetClass() const {};  
      ....
  };
     
   // in the header
class CDerivedClass : public CObject
{
      DECLARE_CLASS(CDerivedClass,CObject)
      
};
 
// in the cpp file
IMPLEMENT_CLASS(CDerivedClass)
      
    
  #define DECLARE_CLASS(TClassType,TSuperClassType)\
     typedef TSuperClassType Super;\
      virtual CClass* GetClass() const override { return GetStaticClass(); }
      static CClass* GetStaticClass();\
      static CObject* _ClassConstructor() { return new TClassType(); }
      
    #define IMPLEMENT_CLASS(TClassType)\
     CClass* TClassType:GetStaticClass()\
     {
       static CClass* ReturnClass = nullptr;
       if(ReturnClass==nullptr)
       {
      	 ReturnClass = new CClass(#TClassType,sizeof(TClassType),AlignOf(TClassType),#_ClassConstructor);
         ... add the properties, etc
        }
      	return ReturnClass;
     }
      

 

Share this post


Link to post
Share on other sites
1 hour ago, AxeGuywithanAxe said:

This reflection system was started from Insomniac Games and expanded over time :

https://github.com/HeliumProject/Reflect

Seems similar to Boost's Hana (though Hana seems less verbose for the user).

 

This does not require the macros for structs/classes (Reflect, Hana): magic_get

This makes me also realize, I should use some POD handles instead of unique_ptr, shared_ptr and ComPtr.

Edited by matt77hias

Share this post


Link to post
Share on other sites

The systems like Hana can help provide a few things, but ultimately the compilation model is not amenable for reflection and invocation. 

Under the traditional compilation model that has been around since the 1960s and were incorporated in the original tools, everything is about small and tight.  Functions are inlined, unused elements are completely elided, template entries may never be generated, link-time optimizations may merge and strip functions. The compilers and linkers do everything they can to reduce and remove the information at every level, exactly the opposite of what reflection needs.

Incorporating all that information so other code can use it with reflection requires significant work.  Adding a bunch of interfaces through the standard compilers (usually meaning macro magic or template magic) usually does pretty good but is not perfect. To get it working everywhere requires a fundamental change to the compilation and linking model, like Microsoft did with C++/CLI compilation. They target the CLI which preserves all those elements rather than a tightly-compiled executable that removes them wherever possible.

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


  • Advertisement