Lets assume we have some collections:

struct A { virtual ~A(); };
struct B : A {};
struct C1 : B {};
struct C2 : B {};
struct C3 : A {};

std::vector< C1 > c1s; 
std::vector< C2 > c2s;
std::vector< C3 > c3s;

Is it now possible to create a Handle template class that access c1s, c2s, c3s via an index?

Handle< C1 >, Handle< C2 > and Handle< C3 > are trivial. But is it somehow possible to cast Handle< C1 > and Handle< C2 > to Handle< B > while still accessing the right vectors (though returning B* or B&)? Similarly, is it somehow possible to cast Handle< C1 >, Handle< C2 >, Handle< C3 > and Handle< B > to Handle< A > while still accessing the right vectors (though returning A* or A&)? (Mis)using std::type_index?

I typically have component handles act less like stand-alone pointers and more like indices (which require some external knowledge of the container being indexed). Often if one system of components is referencing another system of components, it can have a pointer to the other system, while the components simply hold index-like handles (instead of pointer-like handles).

The system can dereference a component's index-like-handle by combining it with it's own pointer. To catch programming errors, in debug builds, my index-like handles contain an index and a pointer/ID as well, so that the dereference process can be validated (is this handle being used to index the correct container?).

Off topic:

31 minutes ago, matt77hias said:

struct A { virtual ~A(); };

In this example, that just forces C1/C2/C3 to contain a vtable for absolutely no reason at all. Firstly, inheritance is a code smell, and secondly, A is not being used as an interface for ownership (there's no calls to delete with an A* argument).

31 minutes ago, matt77hias said:

But is it somehow possible to cast Handle< C1 > and Handle< C2 > to Handle< B > while still accessing the right vectors (though returning B* or B&)? Similarly, is it somehow possible to cast Handle< C1 >, Handle< C2 >, Handle< C3 > and Handle< B > to Handle< A > while still accessing the right vectors (though returning A* or A&)?

This feels vaguely like a violation of OO's LSP rule to me for some reason (any algorithm that works on a base type must also work on a derived type)....

But yes, Handle<B> could contain a pointer to the std::vector as well as the index into it... This gets complicated beecause the vectors are of different types. One (probably heavy-handed) solution off the top of my head would be for Handle<B> to contain an accessor function like:
std::function<B*()> getter = [&c2s, c2index]() { return &c2s[c2index]; }

Alternatively, Handle<B> still just contains an index and a templated getter function:

template<> struct Handle<B>
{
  int index;
  template<class T>
  B& Dereference(std::vector<T>& vec) const { return vec[index]; }
}

And as above, in debug builds I would store extra data inside the handle to validate that the correct vector is being passed into the Dereference function.

8 minutes ago, Hodgman said:

std::function<B*()> getter = [&c2s, c2index]() { return &c2s[c2index]; }

Interesting.

9 minutes ago, Hodgman said:

The system can dereference a component's index-like-handle by combining it with it's own pointer. To catch programming errors, in debug builds, my index-like handles contain an index and a pointer/ID as well, so that the dereference process can be validated (is this handle being used to index the correct container?).

I currently pass indices around, but that does not feel right. It exposes way to many internals. On my scene's side, I have something like this:

template< typename ElementT, typename... ConstructorArgsT >
ID Create(ConstructorArgsT&&... args);

template< typename ElementT >
ElementT &Get(ID id) noexcept;

template< typename ElementT >
const ElementT &Get(ID id) const noexcept;

template< typename ElementT >
size_t GetNumberOf() const noexcept;

template< typename ElementT, typename ActionT >
void ForEach(ActionT action);

template< typename ElementT, typename ActionT >
void ForEach(ActionT action) const;

Here, ID is a typedef for size_t. This approach forces the user to chain Create -> Get with the same template type for instance. Adding type information as wel to the ID, does not feel right (because I expect some type enum and some switch/case statements to appear). The largest problem, however, is the getter: it returns a reference. You may use the reference, but not store it. If the std::vectors are enlarged, all references and pointers are dirty. Only the IDs will stay the same.

I also took a look at the LumixEngine which does not expose references or pointers at all, but uses the "ID" for all operations, resulting in some giant monolithic universe/scene interface.

At the side of my Entity, I have something like this:

ID m_parent;

std::vector< ID > m_childs;

std::unordered_multimap< std::type_index, ID > m_components;

It requires some book keeping as well with regard to the template parameter. Therefore, I thought to use some handle which basically works as a pointer.

Alternatively, I can keep working with IDs internally and create an Handle< C > anyway for only the most derived types.

so

template< typename ComponentT >
ID Get() const noexcept;
template< typename ComponentT >
std::vector< ID > GetAll() const;

becomes

template< typename ComponentT >
Handle< ComponentT  > Get() const noexcept;
template< typename ComponentT >
std::vector< Handle< ComponentT  > > GetAll() const;

I just do not allow to use the base Component type anymore.

29 minutes ago, Hodgman said:

In this example, that just forces C1/C2/C3 to contain a vtable for absolutely no reason at all. Firstly, inheritance is a code smell, and secondly, A is not being used as an interface for ownership (there's no calls to delete with an A* argument).

vtable: All my components inherit a single Component class providing some basic but very limited functionality. So even though, I put everything on the stack in the sub classes, a virtual destructor is cleaner.

inheritance: apart from the Component base, I sometimes add some sub bases such as Camera and Light as well.

Summary:

If I drop handle polymorphism. My scene could provide:

Handle< Model > Import(const ModelDescriptor &desc);
Handle< Model > Import(const ModelDescriptor &desc, std::vector< Handle< Model > > &models);

template< typename ElementT, typename... ConstructorArgsT >
Handle< ElementT > Create(ConstructorArgsT&&... args);

template< typename ElementT >
Handle< ElementT > Get(ID id) noexcept;
template< typename ElementT >
Handle< const ElementT > Get(ID id) const noexcept;

template< typename ElementT >
size_t GetNumberOf() const noexcept;

template< typename ElementT, typename ActionT >
void ForEach(ActionT action);
template< typename ElementT, typename ActionT >
void ForEach(ActionT action) const;

My entity could provide:

template< typename ComponentT >
Handle< ComponentT > Get() const noexcept;
template< typename ComponentT >
std::vector< Handle< ComponentT > > GetAll() const;

template< typename ComponentT >
void AddComponent(Handle< ComponentT > component);

template< typename ActionT >
void ForEachComponent(ActionT action);
template< typename ActionT >
void ForEachComponent(ActionT action) const;

The user only sees a Handle< Entity > or Handle< SpecificComponent >, while I keep working internally with IDs.

The alternative is letting the user do all conversions himself (with the right type).

1 hour ago, Shaarigan said:

According to @Hodgman's post, I myself use handles (indexing) especially then when I want to manage inconsistent data in a continous data chunk like for .ini/.json files or DOM trees so that grouping nodes into parent/child relations is more straight forward and there arent ever fixups needed when grow/copy the buffer from one location into another

For serialization/serialization, this is nice. But it can be very prone to errors. Internally, I want to use indices anyway, but maybe I can encapsulate this for the outside? Now my handles are plain integers (size_t), containing no type info or whatsoever.

My Handle< T > will be very similar to an index. It just overrides operator*, operator-> by accessing the right collections.

2 hours ago, Shaarigan said:

do you really need more than 4.294.967.295 fields??

Probably not, but for now I am not going to make all integers as small as possible. std::vector< T >::size has return type size_t, so I use that for now.

2 hours ago, Shaarigan said:

encapsulate a reference to the container

I redirect to my scene methods based on the handle type:

template< typename ElementT >
Handle< ElementT > Get(ID id) noexcept;

I was also hoping for a way to do some really dynamic casting based on the std::type_info. But this will never be possible in a statically typed language such as C++. The only problem that I still face is how to iterate the Components of an Entity? The Entity stores only IDs without a type. Storing the handles as well is not possible since you cannot put Handle< T > and Handle< U > in the same collection (therefore, I originally thought of using the base Handle< Component >). When I was still using Component pointers instead of IDs, I could just iterate these pointers. I use this functionality to pass messages and state (i.e. if the Entity is made inactive, all component are inactive as well). For state information, it is possible to gather that information when needed by accessing the owning Entity from the Component, but that is unnecessary overhead. I could also delegate this functionality to the scene, which iterates all Components and notifies those who belong to the given Entity, but that is unnecessary overhead as well. The cleanest way would be notifying the Components from the Entity itself. 

On 12/12/2017 at 12:18 PM, Hodgman said:

One (probably heavy-handed) solution off the top of my head would be for Handle<B> to contain an accessor function like:
std::function<B*()> getter = [&c2s, c2index]() { return &c2s[c2index]; }

Experimented with this today. This is a really good and robust solution! The only "drawback" is the "huge" size of std::function.

Everyone that wants to try it online, be my guest: https://wandbox.org/permlink/zkmVbcyXxYFu942i

#include <functional>
#include <iostream>
#include <memory>
#include <vector>

namespace experimental {
    struct Component { 
        virtual ~Component() = default;
        virtual void Test() const noexcept {
            std::cout << "Component::test()" << std::endl;
        }
    };
    struct Camera : Component {
        virtual void Test() const noexcept {
            std::cout << "Camera::test()" << std::endl;
        }
    };
    struct Light  : Component {
        virtual void Test() const noexcept {
            std::cout << "Light::test()" << std::endl;
        }
    };

    std::vector< Camera > g_cameras; 
    std::vector< Light >  g_lights;

    template< typename T >
    T &Get(size_t index) noexcept;
    template<>
    inline Camera &Get< Camera >(size_t index) noexcept {
        return g_cameras[index];
    }
    template<>
    inline Light &Get< Light >(size_t index) noexcept {
        return g_lights[index];
    }

    template< typename T >
    struct H final {
  
        explicit H(std::function< T *() > getter) 
            : m_getter(std::move(getter)) {}
    
        H(const H &h) = default;
        H(H &&h) = default;
    
        template< typename U >
        H(const H< U > &h) 
            : m_getter(h.m_getter) {}
        template< typename U >
        H(H< U > &&h) 
            : m_getter(std::move(h.m_getter)) {}
    
        ~H() = default;
    
        H &operator=(const H &h) = default;
        H &operator=(H &&h) = default;
    
        explicit operator bool() const noexcept {
            return nullptr != Get();
        }
    
        T &operator*() const { 
            return *Get(); 
        }
        T *operator->() const { 
            return  Get(); 
        }
        T *Get() const { 
            return m_getter(); 
        }
    
        std::function< T *() > m_getter;
    };
    
    template< typename U >
    static H< U > Create(size_t index) {
        return H< U >([index]() { 
            return index == 0 ? nullptr : &experimental::Get< U >(index - 1); 
        });
    }
}

using namespace experimental;

int main() {
    // Sizes
    std::cout << "Size of a raw    ptr: " << sizeof(Component *) << std::endl;
    std::cout << "Size of a unique ptr: " << sizeof(std::unique_ptr< Component *() >) << std::endl;
    std::cout << "Size of a shared ptr: " << sizeof(std::shared_ptr< Component *() >) << std::endl;
    std::cout << "Size of a handle ptr: " << sizeof(H< Component *() >) << std::endl;
    
    // Populate
    g_cameras.emplace_back();
    g_lights.emplace_back();
    
    H< Component > camera = Create< Camera >(1);
    H< Component > light  = Create< Light >(1);
    camera->Test();
    light->Test();
}