18 hours ago, Kylotan said:

My point above stands. This particular update requires accessing 3 distinct parts of memory within the same loop, just to perform a relatively trivial operation. Having the data all co-located inside each Entity object, and allocating Entities from a pool so they are located near each other would have simpler code and probably perform better as well.

That true, and a toy project is not good proof. But the point is to solve that isue of bad use of caches and CPU prefetching. Is fall back prety fast to OOP or braking DOD rulez.

the point is to move on into advance solution. In DOD it might be done differently. Posibly counter intuitive. But probaly doable.

First thing in mind.

keep DoD pure.

EntityID Key. Take out use of pointers make Entity and components moveable in the array.

that gives many way to sort and move to other systems

Position used bij few Systems

one pure of position PODs

other POD including position and orientation

other POD inc. position and velocity.

That a direction to pioneer into.

Reusing and sorting Component 

if entity start moving the positionPod and movable is put in pod incl position and velocity in other system for these PODs

i think this is

normalizing tabels

by having multiple tables without gaps.

excistance based programing. It also means iets system act like booleaan no check for Null. As no pointers.

That direction solution could be viable.

19 hours ago, Kylotan said:

My point above stands. This particular update requires accessing 3 distinct parts of memory within the same loop, just to perform a relatively trivial operation. Having the data all co-located inside each Entity object, and allocating Entities from a pool so they are located near each other would have simpler code and probably perform better as well.

It depends on the data access pattern. If most entities need position/rotation/velocity, just put them together.

struct CommonData
{
	vec3 position;
	vec3 scale;
	quat rotation;
	
	vec3 velocity;
};

CommonDataSystem
{
	Array<CommonData> data[MAX_ENTITY_COUNT];
	CommonData& Lookup(Entity);
};

If only some entities need a run-time name, put those into a separate HashTable.

struct Name
{
	char Value[32];
};

NameSystem
{
	Hash<Entity, Name> names;
};

I have only implemented two variants of these, those were initially a couple of hours of work that was fairly representative in terms of performance to a fully featured implementation.

I'd approach the problem by designing an agnostic API with a stub implementation, and then brute-force choose the best performing one once I have a non trivial project going. 

This is strictly only impossible if the implementation has strong implications for how programmers must design the resulting systems that use that particular implementation. I'm not sure what would even fit the "strong" qualification there. Even the variant where the components stored with systems (and the programmer who writes those systems has to put them there) can probably be circumvented with some macros or code introspection / code generation (hello clang libs, until C++ finally has a proper metaprogramming system) that says "I need physics and renderer related components here" and then figure out where to put that stuff based on the dependency graph of all systems that need it. 

I guess if you really want to you can probably circumvent all of those problems by using some kind of DSL. 

What am I missing?

13 hours ago, agleed said:

"I need physics and renderer related components here"

That would be very flexible and high performance at the same time.

I believe it is possible by using template trick to notify (+custom type id manipulator) e.g. :-

ECSUtility::setPlaceNearby<Component_graphic,Component_physic>()

Sadly, I can't find any concrete example.  .... It is not a trivial task.

On 8/29/2017 at 4:20 PM, Kylotan said:

Doesn't look great to me, given that there are 3 distinct areas of memory that need to be accessed  ...

I improved my engine from 3->2, and got performance boost ~60%.  Thank!!

Do you generally pass entity around as (1) an entity OR (2) a component pointer?

The first is more flexible approach, but query "entity->component pointer" costs me a lot.  (need 2 distinct area)

The second is less flexible.  It need only 1 distinct area.

Are there any other approaches?  

By the way, your posts help me a lot.   Do you write some books?  I've visited your website ( http://blog.ebonyfortress.com/), but find none.

On 9/1/2017 at 2:22 PM, Finalspace said:

Potential components like physics/render-component which should be processed contigously has its own vector,

I agree with the whole post.  Physic-Graphic pair that I mentioned is not a good example. 

In sometimes, I love to have this feature to easily get some performance boost from data locality.  

For example, in the last cycle of development, I may discover that 2 certain components are always access together.   If an entity has one component, it always has the other one. 

The example (PhysicsSystemItem) is interesting.    Thank.

On 9/1/2017 at 3:34 PM, LorenzoGatti said:

Just released: Boost.PolyCollection, an interesting point of view on a related problem: http://www.boost.org/doc/libs/1_65_0/doc/html/poly_collection.html

I took a look at it. Never know such thing exist.   Thank.

I don't generally use explicitly component-based entities for my work, so I don't have much to recommend in that area. I prefer a more traditional entity object, with data inline, but migrated out to ad-hoc components where needed. I generally believe in writing the game, and then changing the code to optimise later.

But I am never interested in supporting 30,000 entities, so hyper-performance issues don't affect me.