@Oberon_Command

27 minutes ago, Oberon_Command said:

An advantage to having "entities" be only simple identifiers - handles, really - is that you can make some component state inaccessible to other components and when you do need that access, the dependency is obvious because you have to tell the code that wants access where to find the components. That makes the code more verbose, but that's a feature, not a bug. Requiring explicit dependency specification is a feature, not a bug. Without that, it can become difficult to reason about what depend on which components. An inability to reason about dependencies can lead to a maintenance nightmare.

Would you be able to provide a quick pseudo code example of what your talking about here? 

1 hour ago, boagz57 said:

@Oberon_Command

 

Would you be able to provide a quick pseudo code example of what your talking about here? 

void move_character(
  Transform& transform,
  const CharacterDefinition& definition,
  const CharacterInputState& input,
  float dt)
{
  const Vector desiredMoveDirection = input.get_desired_move_direction();
  const Vector movement = definition.movementParameters.compute_translation_vector(desiredMoveDirection, dt);
  transform.translate(movement);
}

void move_all_characters(
  SparseArray<Transform>& transforms, 
  const SparseArray<CharacterDefinition>& characters,
  const SparseArray<CharacterInputState>& inputStates,
  float dt)
{
  for (const CharacterInputState& input : inputStates.get_active_set())
  {
    const EntityHandle entity = input.get_handle();
    const CharacterDefinition* definition = characters.try_and_get(entity);
    Transform* transform = transforms.try_and_get(entity);
  
    if (definition && transform)
    {    
      move_character(*transform, *definition, input, dt);
    }
  }
}

Then we can call move_all_characters from a place where we have access to the transforms, characters and input states, which is probably glue code that binds all this stuff together. Of course, we don't actually need these to be sparse arrays, they could be some other kind of abstraction. Maybe even an opaque structure that lives off in memory somewhere and we don't actually know what it is, apart from the fact that we can call a function on it (or a reference or smart pointer pointing to it) that gives us a CharacterDefinition given an EntityHandle.

This also allows you to have entirely different sets of entities for different kinds of "game objects" if some entities have entirely disjoint sets of components, rather than forcing all components to exist in the same set of entities. Of course, you can do that with other frameworks, too, but here that idea becomes very easy to implement.

3 hours ago, boagz57 said:

So if I try and think of components more broadly then I'm sure other components will inevitably start needing the functionality of other components. @Servant of the Lord suggests passing in components to other components to help with dependent functionality. You were saying that components needing other components should just go through the Entity. I know you described this a little bit but would you be able to give a quick pseudo code example? And would this be any better than just adding components as parameters in other component functions/constructors?

Components wouldnt necessarily need to go through the entity to access other components.  A component could very easily have a pointer to another component if needed.  This would avoid the process of getting the entity and then querying for the other component.  In that case, I'd suggest using std::shared_ptr and std::weak_ptr.  A component can keep an std::weak_ptr to another component if needed.

As far as pseudo-code for getting components through the entity, that's simple:

TransformComponent* T = GetEntity().GetComponent<TransformComponent>();
if(T)
{
  	// do stuff here
}

The thing is, this all depends on how you want to organize your code.  As you can see in this thread, some people will like to do things one way and others another way.   And, people tend to think if you do it a different way from what they're used it, it's going to be awful and full of problems.  But the reality is that each way has it's benefits and drawbacks. The problems, IMO, usually come because you either 1) chose an approach that's not well suited to your problem, or 2) dont follow that approach correctly and end up with messy and hacky code.  The example earlier of having a Model Component with a position, when you already have a Position component, is such a thing.   When architecting these systems, it's very important to carefully think through when you add new stuff, to make sure it follows the established rules and assumptions of the architecture.  A lot of times you end up with something you think cant be done, or something you think can be done but only if you break the rules or make exceptions.  But, that usually means you just havent thought about it in the right way.   If your architecture is clean and well thought out, you can almost always find ways to solve problems that fit within that architecture.

ECS is very powerful, but it requires you to really think about what will be components, what granularity you need, what problems you have and what features you will actually need.  You need to break down your game and features, and then think of logical ways to break them down into components.

@0r0d

6 hours ago, 0r0d said:

The thing is, this all depends on how you want to organize your code.  As you can see in this thread, some people will like to do things one way and others another way.   And, people tend to think if you do it a different way from what they're used it, it's going to be awful and full of problems.  But the reality is that each way has it's benefits and drawbacks.

This is one of the major problems I'm having with trying to design my code base. Deciding when something needs to redesigned. I've heard all the horror stories about software going down a certain path and then after a while having problems that require massive amounts of work to try and fix. Every software article/book you read is almost entirely about how you should try and avoid these kinds of problems. Because of this I'm constantly second guessing architecture and spotting potential issues here and there. I guess programming is about finding the right balance between clean code and moving forward despite obvious issues. The problem is I rarely see more pragmatic discussions on developing and designing software architecture which can make it difficult to develop in my day to day software design.

If anyone has any advice on this aspect I would love to hear it. That is how do you typically design an architecture and balance the need to constantly refactor your code (due to obvious potential issues) and the need to move forward in a project? Does it make sense to treat refactoring like optimization? That is, only refactor when any of the 'potential issues' really do start effecting your code base? Or is that too much of a risk as this can leave your architecture a mess after too long.

12 hours ago, boagz57 said:

So if I try and think of components more broadly then I'm sure other components will inevitably start needing the functionality of other components. @Servant of the Lord suggests passing in components to other components to help with dependent functionality.

Just to clarify, since I kinda glossed over that earlier - in my mindset, components are almost pure data (albeit sometimes with minor functions), and *data* doesn't have dependencies, *code* does. A system can have dependency on more than one component.

If a *system* requires more than one component, you can pass them both in to that system. For example, let's say you had a Transform component, containing position/rotation/etc... And you had a Collision component, containing a bounding box relative to Transform, and a Animation component that is drawn at Transform's position.

There is no "Transform" system**. But PhysicsSystem can be handed the Collision comp and Transform comp, and GraphicSystem can be handed the Animation comp and the Transform comp.

In the ECS mindset I subscribe to, Entity is just a concept, identified by an EntityID, systems operate on components en-bulk, and so you'd pass in entire arrays of components to the systems that need that type of component. It'd look roughly something like this:

//Systems that only need to read a specific component type (e.g. GraphicSystem's need of Transform), can take a const reference to that array.
//This is benenficial for debugging, ofcourse, but also for paralization, if actually needed.
//Some systems may have private internal arrays of components mapped to IDs, for bookkeeping and optimizations and such.

PhysicsSystem physicsSystem(&transformComponents, &collisionComponents,
                                staticWorldGeometry /* or whatever other components or non-component stuff this system needs */);
GraphicSystem graphicSystem(&transformComponents, &animationComponents, &particleComponents, &textureCache, &shaders, &etc...);

This seems like a very different architecture then what you are doing, and is alot of boilerplate for benefits most games don't need. Simple entities and composition (and possibly inheritance) is much superior in most cases.

@Servant of the Lord Great! Thanks for clarifying and being very active on this post.

I think I'm confused all of the sudden.  So to set my starting point out explicitly to see if I am missing something, here's a specific randomly imagined set of game components.

The entities are of course completely dumb "objects" that exist only to do the following - have an ID, have a totally generic list of components, and perhaps have a list of entities (unless possession of child entities is a component itself - but my purpose I'll assume I didn't implement it that way), and also of course to be serviceable/savable.

The following component / behavior interfaces may have been create:  Location (includes position and orientation), Inertia (includes mass & vector), Goals (needs, wants, personality info), Status (Health & Other Similar State), Senses (a base component for subtypes like: Sight (parameters about observing the world visually - aka ), Hearing (parameters about observing the world, etc), Controller (a base component with multiple subtypes like: CommandController (support for accepting commands as triggers to action), InputController (support for mapping input to commands), ScriptController (support for attaching 1 or more behavioral scripts)), Metabolism (a component to manage tracking actions taking "energy", and time or materials restoring it - simplified).

Now it should be easy enough to see, that in any complex object graphs, for instance things like War Wagons or Tanks.  There will be multiple non-trivial relationships between specific subsystems and their parent parent systems.  And these things aren't just wired up by "find the COMPONENT_TYPE of my PARENT/CHILD and use it"  These are EXPLICIT relationships.  The left tank tread has relationships to the 4 wheels on the left side of the tank, which in turn connect to 1 or more drive shaft, which is the same drive shaft as connected to the right side.

So the whole point of ECS in my mind is that the game ENGINE as a whole operates by walking the tree in appropriate ways to do universal things to EVERY SINGLE COMPONENT of the appropriate TYPE/INTERFACE.  However, no aspect of the actual objects works that same way, but instead would be specific relationship based.

More specific example.  The game engine would loop through/process all controllers, giving them a chance to act.  The game engine would process all Inertia, updating locations.  The game engine would process all metabolism, etc, etc etc.

But the internal code of the individual component, IE the code controller the Tank Engine's "metabolism" (in this case gas consumption) doesn't have code like "this.Entity.GetComponent<GasTank>.Consume(this.CurrentBurnRate)" but instead has code more like "this.CurrentFuelTank.Consume(this.CurrentBurnRate".

Of course I'm not showing code for checking if there isn't enough fuel, triggering reactions to any problems, etc.  I'm just showing the idea that the components have DIRECT EXPLICIT RELATIONSHIP to any other entity or component instances they use.

If we had the rule that an entity could only have 1 component max for each interface, we could change the relationship to components pointing to entities only, instead of some of these relationships being to entities and some being to components - which might be better in some ways (I'll have to think deeper about that part later) ... but either way, the key is that most lookup from 1 place in a an object graph to another are not based on type but base on relationship.

17 minutes ago, Xai said:

I think I'm confused all of the sudden.  So to set my starting point out explicitly to see if I am missing something, here's a specific randomly imagined set of game components.

The entities are of course completely dumb "objects" that exist only to do the following - have an ID, have a totally generic list of components, and perhaps have a list of entities (unless possession of child entities is a component itself - but my purpose I'll assume I didn't implement it that way), and also of course to be serviceable/savable.

The following component / behavior interfaces may have been create:  Location (includes position and orientation), Inertia (includes mass & vector), Goals (needs, wants, personality info), Status (Health & Other Similar State), Senses (a base component for subtypes like: Sight (parameters about observing the world visually - aka ), Hearing (parameters about observing the world, etc), Controller (a base component with multiple subtypes like: CommandController (support for accepting commands as triggers to action), InputController (support for mapping input to commands), ScriptController (support for attaching 1 or more behavioral scripts)), Metabolism (a component to manage tracking actions taking "energy", and time or materials restoring it - simplified).

Now it should be easy enough to see, that in any complex object graphs, for instance things like War Wagons or Tanks.  There will be multiple non-trivial relationships between specific subsystems and their parent parent systems.  And these things aren't just wired up by "find the COMPONENT_TYPE of my PARENT/CHILD and use it"  These are EXPLICIT relationships.  The left tank tread has relationships to the 4 wheels on the left side of the tank, which in turn connect to 1 or more drive shaft, which is the same drive shaft as connected to the right side.

So the whole point of ECS in my mind is that the game ENGINE as a whole operates by walking the tree in appropriate ways to do universal things to EVERY SINGLE COMPONENT of the appropriate TYPE/INTERFACE.  However, no aspect of the actual objects works that same way, but instead would be specific relationship based.

More specific example.  The game engine would loop through/process all controllers, giving them a chance to act.  The game engine would process all Inertia, updating locations.  The game engine would process all metabolism, etc, etc etc.

But the internal code of the individual component, IE the code controller the Tank Engine's "metabolism" (in this case gas consumption) doesn't have code like "this.Entity.GetComponent<GasTank>.Consume(this.CurrentBurnRate)" but instead has code more like "this.CurrentFuelTank.Consume(this.CurrentBurnRate".

Of course I'm not showing code for checking if there isn't enough fuel, triggering reactions to any problems, etc.  I'm just showing the idea that the components have DIRECT EXPLICIT RELATIONSHIP to any other entity or component instances they use.

If we had the rule that an entity could only have 1 component max for each interface, we could change the relationship to components pointing to entities only, instead of some of these relationships being to entities and some being to components - which might be better in some ways (I'll have to think deeper about that part later) ... but either way, the key is that most lookup from 1 place in a an object graph to another are not based on type but base on relationship.

 

I think having explicit dependencies like this is reasonable, and something I use in my own engine.  For example, my CharacterController component has pointers to the RigidBody and AnimMesh components. 

9 hours ago, boagz57 said:

@0r0d

This is one of the major problems I'm having with trying to design my code base. Deciding when something needs to redesigned. I've heard all the horror stories about software going down a certain path and then after a while having problems that require massive amounts of work to try and fix. Every software article/book you read is almost entirely about how you should try and avoid these kinds of problems. Because of this I'm constantly second guessing architecture and spotting potential issues here and there. I guess programming is about finding the right balance between clean code and moving forward despite obvious issues. The problem is I rarely see more pragmatic discussions on developing and designing software architecture which can make it difficult to develop in my day to day software design.

If anyone has any advice on this aspect I would love to hear it. That is how do you typically design an architecture and balance the need to constantly refactor your code (due to obvious potential issues) and the need to move forward in a project? Does it make sense to treat refactoring like optimization? That is, only refactor when any of the 'potential issues' really do start effecting your code base? Or is that too much of a risk as this can leave your architecture a mess after too long.

You get better at deciding what architecture you need and when to redesign with experience.  If you dont have a lot of experience with these things, then listening to a lot of contradictory opinions will just confuse you.  I would say, listen to people who can tell you how these systems are meant to work and what they're benefits are, then consider that in the context of what you're trying to do, and then make a decision and go with it.  As you implement it you will start to figure out on your own what the benefits and drawbacks are, and how that relates to what you want to do.  I mean, some of the drawbacks might be acceptable to you because they're not a real factor.  If one system is easier to implement but has worse performance, that might be ok with you because performance is not an issue for your game.

So I would say, start by thinking about what you want from your engine.  If you're considering ECS, then break down your needs into what you think your components will be.  You can post it here along with your other requirements for the engine and game.  The main goal is to figure out logical components that dont overlap with each other, and that strike a balance between too small and too broad.  Then when as you keep adding stuff to your engine/game, always think carefully to make sure what you're adding follows the pattern you chose.

18 hours ago, 0r0d said:

There's really nothing wrong with accessing components directly.  It all just depends on what your architecture is trying to accomplish and how it's doing it.  In fact, the entire point of an ECS is that all (or most of) your functionality will be components.  So, for code to do anything useful it has to deal with components.  What's the alternative?  To put functionality into the entity and have that redirect calls to the appropriate component?  That's just going to give you unneeded complexity and problems.   Usually, code shouldnt even need to deal with an entity, since what it should be doing is dealing with the appropriate components.   The entities should mostly only be there as a way to get associated components.

The ECS pattern enables you to express the behavior of entities as a combination of modular components. It doesn't mean the business logic driving this expression is actually contained within the components. If anything, such an approach immediately breaks the modularity and encapsulation you're trying to achieve and can quickly lead to unmaintainable spaghetti code.

Take your AI component and render component as an example. There is now a specific behavioral and component dependency built into the AI component. It's impossible for higher-level code to use the AI component without inheriting this internal behavioral interaction with the render component. The fact that internal ECS behavior can't be decoupled from external game behavior should raise a red flag.

Let's also consider unit testing for a moment. How would one unit test the AI component independently of the render component? The fact that it's impossible should immediately raise another red flag.

There's also the issue of the code itself being coupled at the build level. Your AI component code must now be linked with the render component code. It doesn't matter if the final entity actually has both components. You've created a permanent symbol dependency that must always be resolved either at build-time or load-time (i.e. by the dynamic linker). It might not be a big deal now, but good luck trying to split your components into modular, reusable, domain-specific type libraries.

21 hours ago, 0r0d said:

The problem with the code above is not that you're exposing functionality, it's that you have 2 different components that have a position.  Why is that?  Of course if your components are not orthogonal you will have problems.   There should be no need to set a position on the Model component because it shouldnt have a position in the first place.  If it wants to render, then it should get the Position component by going through the Entity.

Data duplication was a bad example on my part. This is more appropriate to the discussion:

entity.component<Position>().set(10, 10);
if (entity.has<Networking>())
  entity.component<Networking>().markDirty<Position>();

Two completely unrelated components that share only a behavioral relationship (notify networking when position changes)

20 hours ago, Oberon_Command said:

I'm not at all a fan of this approach, or any approach where anything can query any entity for any kind of component from anywhere. It's not far removed from just having global state all over the place. If it's done poorly, it makes for shitty build times, too (if you're using C++).

There's nothing in the code to suggest that any entity can be accessed from anywhere. It's precisely the opposite, where the "move" method accepts only the bare minimum data it needs, and doesn't access any global state. If code shouldn't have access to a particular piece of data (such as an entity ID), then hide it. Data hiding is a separate issue entirely that can be solved using other well-known methods.

The fact that you can query any component type is neither here nor there. If code doesn't have an actual entity ID or other relevant data to work with (because you appropriate hid it), then the knowledge of those types does nothing for you.

20 hours ago, Oberon_Command said:

An advantage to having "entities" be only simple identifiers - handles, really - is that you can make some component state inaccessible to other components and when you do need that access, the dependency is obvious because you have to tell the code that wants access where to find the components. That makes the code more verbose, but that's a feature, not a bug. Requiring explicit dependency specification is a feature, not a bug. Without that, it can become difficult to reason about what depend on which components. An inability to reason about dependencies can lead to a maintenance nightmare.

With a system where entities are only handles, I can make component types that only the code that cares about them needs to care about and as long as I ensure that some kind of mapping from entity to component state exists, I can arrange both the components AND the entities (eg. lists of entities by value!) in whatever the most useful way happens to be for what I'm going to do with them. I don't even need to allow anything outside whatever system deals with that component know what that component is - translation unit-local components can be a thing! For that matter, "components" don't even need to be monolithic objects, they can be "structures of arrays" or even more exotic things as long as there's a consistent interface presented to them that uses the entity handle.

Having entities be identifiers versus objects has nothing to do with component accessibility or dependency specifications. It's also not relevant to the discussion. We could just as easily assume that the entity object in my examples was actually a thin handle/wrapper about a functional interface that stores components in "structures of arrays" (or any layout of your choosing). It doesn't change anything, as it's functionally equivalent.

The only way to truly "hide" a component (or any code for that matter) is to make its symbols inaccessible. This is ultimately determined by how you encapsulate and layer your software into "black boxes", not by any particular implementation. Explicit declaration of dependencies is a nice touch that helps code be more self-documenting and enables certain optimizations in the ESC implementation, sure, but it's useless as a mechanism for controlling type access. I can include the appropriate header file and update the specification at-will to accommodate my new component dependency, and nothing can stop me.

Trying to arbitrarily limit or control type access from code pointless. All you can do is control data, but that's entirely sufficient for all intents and purposes.

21 hours ago, Oberon_Command said:

Whereas in order to make this Unity-style ECS work, you're likely going to need some kind of mechanism to expose arbitrary component types to arbitrary code, which adds more complexity and probably global(ish) state (that can then be abused). Then to make has<T> and component<T> work you probably ALSO need some kind of RTTI mechanism for your components, which extra complexity that isn't always actually necessary for anything other than the code that glues the components together.

These mechanisms already exist, and you use them all the time. Code access and visibility is controlled by its structure and layout, the use of public vs private header files, visible vs hidden symbols in shared libraries, etc. Splitting software components into black boxes that can communicate through public interfaces is nothing new. However that's besides the point. Why does it matter if arbitrary components types are exposed to arbitrary code? Going back to what I said previously, without real data it's a moot point.