Jump to content

  • Log In with Google      Sign In   
  • Create Account

Storing position of member variable


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
13 replies to this topic

#1 Juliean   GDNet+   -  Reputation: 2867

Like
0Likes
Like

Posted 14 July 2014 - 04:23 AM

Hello,

 

I know that there is no such thing like reflection in C++, but I need something a tad bit in that direction. I want to refactor and improve my component system, by automating most loading/serialization/editor-related stuff. For this, I plan on adding a component declaration class, where attributes with a certain known type (int, float, string, ...) are specified. This would look something like that:

class Transform : public ecs::Component<Transform>
{
	math::Vector3 vPosition;
	
	static const ComponentDeclaration& Declaration(void)
	{
		const ComponentDeclaration::AttributeVector vAttributes = 
		{
			{ AttributeType::FLOAT, "x" },
			{ AttributeType::FLOAT, "y" },
			{ AttributeType::FLOAT, "z" },
		}
		
		static const ComponentDeclaration decl(vAttributes);
		
		return decl;
	}
}

Now, I just could have a "serialize/deserialize"-method, but I want to ultimately get rid of those, to minimize work required for each component. Is there any way to calculate/store the "offset" of a member variable and use this to access/set that variable? Something like:

// in the declaration

{ AttributeType::FLOAT, "x", OFFSET(Component::vPosition::x) },

// later on

auto& component = ComponentFactory::CreateComponent(type);

const auto& declaration = component.GetDeclaration();

for(auto& attribute : declaration.GetAttributes())
{
	switch(type)
	{
	case AttributeType::FLOAT:
	{
		float* pAttribute = (float*)(((char*)(void*)(component)) + attribute.offset);
	}
	}
}

How would you do so?

 

PS: Though I don't care about 100% (type)safety and cross-compiler right now, I'm also open for all other, probably better suggestions for that problem you might have.



Sponsor:

#2 Ashaman73   Crossbones+   -  Reputation: 8355

Like
2Likes
Like

Posted 14 July 2014 - 05:02 AM


by automating most loading/serialization/editor-related stuff.

To be honest, I would use only a very simple system, that is , just a de-/serialize (post-deserialization) method which read/write your (primitive) attributes from/to a stream. The overhead to code and maintain a more complex system is not necessary in my opinion and often more error prone than KISS. 

 

Here is a simple example of a more complex scenario, continaing arrays, chunks, complex sub objects etc:

serialize(stream) {
  // handle inventory
  if(inventory.size()>0) {
    stream.write(INVENTORY_CHUNK);
    stream.write(inventory.size());
    for(i=0;i<inventory.size();i++) {
       bool item_present = inventory.get(i);
       stream.write(item_present);
       if(item_present) {
          inventory.get(i).serialize(stream);
       } 
  }

  // handle equipment
  if(equipement.size()>0) {
    stream.write(EQUIPMENT_CHUNK);
    ...
  }

  stream.write(END_CHUNK);
}



deserialize(stream) {
   for( int chunk_id = stream.read();chunk_id!=END_CHUNK; chunk_id = stream.read()) {
      switch(chunk_id) {
        case INVENTORY_CHUNK:
           int size = stream.readInt();
           createNewInventory(size);
           ... continue to read rest ... 

        case EQUIPMENT_CHUNK: 
           // read, but ignore, no longer supported ....
           break;
     }

   }

The benefit is, that you have more control over sub-sequent save game versions and you are able to read and process older file versions more easily without bloating the whole code. The latter has the benefit, that you can add new attributes more easily or ignore obsolete one, without forcing a new game file version. It is really annoying if almost every single bug-fix will make older save-games obsolete.


Edited by Ashaman73, 14 July 2014 - 05:06 AM.


#3 Juliean   GDNet+   -  Reputation: 2867

Like
0Likes
Like

Posted 14 July 2014 - 05:15 AM


To be honest, I would use only a very simple system, that is , just a de-/serialize (post-deserialization) method which read/write your (primitive) attributes from/to a stream. The overhead to code and maintain a more complex system is not necessary in my opinion and often more error prone than KISS.

 

While I'm grateful for the feedback, I have to disagree. This system is in use now for almost two years, and the need for a more complex system arouse just lately after I found that my first, simple system didn't work out so well after a long time - the overhead for adding one single component is way too much.

 

I must admit that I left some information out in order to explain the problem simplier, but its not only serialization. There is also a few other key points that make a system like I proposed pay off:

 

- Editor functionality. Right now, I have to create at least 3 different "controller"-classes for each new component, basically just copying & modifying from any other component. Such a "reflection"-system would automate that, and allow me to create even more "complex" editor gui controls for the component without N-overhead (N = number of components).

- Cross-Plugin-Interaction. My engines actual game code is made out of plugins, and components are mutually declared in those plugins. Some plugins might want to use certain aspects (read: attributes) of any other plugins components, or instantiate it, w/o having the code. Right now thats a free-for-all, that attribute-system would give a much safer interface for that interaction.

- There is probably more which I can't just remember right now.

 

So again, I much appreciate the input, but trust me, I know very well what I want to do and why I want it ;)

 

Any solution for the original problem though?

 

EDIT: If found something that appearently does what I want to do, offsetof. Seems not to be designed to work with non-POD-classes, but appearently works on MSVC, thats fine, portability is not a real requirement yet. I'm still looking for a way to make its usage a bit easier. Right now I'd have to do:

class Transform : public Component<Transform>
{
    math::Vector3 vTest;

    void Test(void)
    {
           const size_t offset = offsetof(Transform, vTest.x);
    }
}

I basically want to get rid of the need for specifying the class name everytime. I though about creating a templated protected member-variable in the base-class, which already has the derived class as template-argument:

template<typename Derived>
class Component
{
protected:

	template<typename Type>
	size_t Offset(Type Derived::* pMember) const
	{
		return offsetof(Derived, pMember);
	}
}

But this won't work for the case of the vector, where the actual variable is stored inside another struct in the component, whereas the pure "offsetof"-macro works. Any ideas if this can be fixed? Ok, this actually doesn't work at all, any ideas for a working solution?


Edited by Juliean, 14 July 2014 - 05:41 AM.


#4 belfegor   Crossbones+   -  Reputation: 2737

Like
3Likes
Like

Posted 14 July 2014 - 05:36 AM

Recently i have found interesting article where the author is using "pointer to a member" (with magic template meta-programming) to get that offset.

I think it is somewhat related to what you are asking, but i am still trying to grasp this myself how all this works so i am not 100% sure if it fits your needs.



#5 Juliean   GDNet+   -  Reputation: 2867

Like
0Likes
Like

Posted 14 July 2014 - 05:47 AM


Recently i have found interesting article where the author is using "pointer to a member" (with magic template meta-programming) to get that offset.

I think it is somewhat related to what you are asking, but i am still trying to grasp this myself how all this works so i am not 100% sure if it fits your needs.

 

Actually, pointer to member seems quite right, as you can see I've already been trying to use it in my example above, but didn't quite get it totally. Since I have both the class and member type, I can actually store the pointer-to-member as a void* alongside the declaration data, and cast it back when I want to use it... at least I hope this works the way I think about it, I'll at least give it a try.

 

EDIT: Unfortunately, pointer-to-member seems to also have a problem with the child-POD-struct:

class Sprite : 
	public ecs::Component<Sprite>
{
public:
	math::Vector2f vPos;
	float z;
	
	void Test(void)
	{
		float Sprite::* pZ = &Sprite::z; // works
		float Sprite::* pX = &Sprite::vPos::z; // doesn't even compile... I don't even know how that should work
	}
};

I don't know... is there even a solution to this? offsetof would work, though I like pointer-to-member better... I don't want to replace the vector with pure floats eighter... any ideas?


Edited by Juliean, 14 July 2014 - 05:56 AM.


#6 AllEightUp   Moderators   -  Reputation: 4322

Like
3Likes
Like

Posted 14 July 2014 - 06:29 AM

You might want to look at doing this in a more C++ like manner without the potentially difficult to deal with side effects of pointers into protected/private data.  The way I went recently was to implement a system like this using std::function.  Basically using accessors I could bind up access in nice little objects which worked with my serialization.  So, for instance, in your given example, I wouldn't expose x, y, z separately I'd simply bind the accessor to the vector as:

 

std::function< Vector& () > accessor( std::bind( &MyComponent::Position, std::placeholders::_1 ) );

 

With that, pop it in a map of named accessors and you can get/set the component data without breaking encapsulation of the class with evil memory hackery.  Obviously there is more work involved in cleaning this up to allow multiple types, separate get/set functions and a bunch of other things I wanted supported but it is considerably better behaved than direct memory access.  The primary reason I avoided the direct memory access was because I have a number of items which need to serialize but work within a lazy update system, if I bypass the accessor, the data in memory can be in completely invalid states.  With a bound member (or internal lambda), everything can be runtime type safe (unlikely to get compile time type safety), works with lazy update systems and generally a much more robust and less hacky solution.


Edited by AllEightUp, 14 July 2014 - 06:32 AM.


#7 swiftcoder   Senior Moderators   -  Reputation: 11082

Like
2Likes
Like

Posted 14 July 2014 - 08:10 AM


Since I have both the class and member type, I can actually store the pointer-to-member as a void* alongside the declaration data, and cast it back when I want to use it... at least I hope this works the way I think about it, I'll at least give it a try.

This is not safe.

 

There is no guarantee that a pointer-to-member is the same size as a regular pointer, which will lead to some interesting results if you cast between them.


Tristam MacDonald - Software Engineer @Amazon - [swiftcoding]


#8 tivolo   Members   -  Reputation: 1029

Like
1Likes
Like

Posted 14 July 2014 - 08:17 AM

 

 

EDIT: Unfortunately, pointer-to-member seems to also have a problem with the child-POD-struct:

class Sprite : 
	public ecs::Component<Sprite>
{
public:
	math::Vector2f vPos;
	float z;
	
	void Test(void)
	{
		float Sprite::* pZ = &Sprite::z; // works
		float Sprite::* pX = &Sprite::vPos::z; // doesn't even compile... I don't even know how that should work
	}
};

I don't know... is there even a solution to this? offsetof would work, though I like pointer-to-member better... I don't want to replace the vector with pure floats eighter... any ideas?

 

 

The problem is that &Sprite::vPos::z is not a pointer-to-member of class Sprite, but rather a pointer-to-member of math::Vector2f, which is entirely unrelated to Sprite.
One solution to your problem would be to implement a generic (but typesafe!) pointer-to-member, something using inheritance and type erasure should do the trick.



#9 Juliean   GDNet+   -  Reputation: 2867

Like
0Likes
Like

Posted 14 July 2014 - 08:29 AM


You might want to look at doing this in a more C++ like manner without the potentially difficult to deal with side effects of pointers into protected/private data. The way I went recently was to implement a system like this using std::function. Basically using accessors I could bind up access in nice little objects which worked with my serialization. So, for instance, in your given example, I wouldn't expose x, y, z separately I'd simply bind the accessor to the vector as:

 

Hm, definately interesting, haven't looked at it that way before. There is at least no problem with private data here, because components are required to have all their data public for the systems to work with them. They are in fact just "POD"-data containers with a few helper functions brought in by the base classes. So there wouldn't be any substantial problem with direct memory exposure, at least none I can think of. I actually do have a few attributes that use a "dirty"-flag when they are changed, this would actually be a good point for an accessor. I don't really like the std::function-systax, but I quess I can hide that behind some overloaded methods. I'll still go with seperate attributes, mainly because they need to be named in the editor interface, which could range from x/y/z to yaw/pitch/roll, to almost everything one can think of. Thanks for the suggestion, I'll definately try it out and compare it to the direct-memory-access, which one works easier/better I'll take.

 


There is no guarantee that a pointer-to-member is the same size as a regular pointer, which will lead to some interesting results if you cast between them.

 

Alright, so I can forget that then. Don't know how else I'd get that to work for multiple types

 


The problem is that &Sprite::vPos::z is not a pointer-to-member of class Sprite, but rather a pointer-to-member of math::Vector2f, which is entirely unrelated to Sprite.
One solution to your problem would be to implement a generic (but typesafe!) pointer-to-member, something using inheritance and type erasure should do the trick.
.

 

Hm alright, thats another option. I'm tempted to try it out, lets see what happens.

 

Thanks all so far, I'll get back when I got any more problems.



#10 SeanMiddleditch   Crossbones+   -  Reputation: 7748

Like
2Likes
Like

Posted 14 July 2014 - 03:23 PM

Typically you need a layer of indirection to be present. Something like an IMemberInfo class that can then be specialized for the different types of pointer-to-member, getter/setter, etc. You also need to be careful with pointer adjustments for base classes when multiple inheritance is in play, including multiple inheritance of interfaces. This means you also need some kind of list of ClassInfo to that maps a particular class to its base including pointer adjustments. Something like:

class ClassInfo;
class IMemberInfo;

struct BaseInfo {
  ClassInfo* classInfo = nullptr;
  ptrdiff_t offset = 0;
};

class ClassInfo {
  flat_map<string, unique_ptr<IMemberInfo>> _members;
  vector<BaseInfo> _bases;

public:
  bool Set(string const& memberName, void* object, const void* data);
  bool Get(string const& memberName, const void* object, void* outData);

  void AddMember(string memberName, unique_ptr<IMemberInfo> member);
  void AddBase(ClassInfo* classInfo, ptrdiff_t offset);
};

class IMemberInfo {
public:
  virtual ~IMemberInfo() = default;
  virtual void Set(void* object, const void* data) const = 0;
  virtual void Get(const void* object, void* outData) const = 0;
};

template <typename C, typename M>
class DirectMemberInfo {
  C::*M _member = nullptr;

public:
  DirectMemberInfo(C::*M member) : _member(member) {}

  void Set(void* object, const void* data) const override {
    static_cast<C*>(object)->*_member = *static_cast<const M*>(data);
  }

  void Get(const void* object, void* outData) const override {
    *static_cast<M*>(outData) = static_cast<C*>(const object)->*_member;
  }
};

bool ClassInfo::Set(string const& memberName, void* object, const void* data)
{
  auto member = _members.find(memberName);
  if (member != _members.end()) {
    member->second->Set(object, data);

  for (auto base : _bases) {
    auto adjusted = static_cast<char*>(object) + base.offset;
    if (base.classInfo->Set(memberName, adjusted, data))
      return true;
  }

  return false;
}

bool ClassInfo::Get(string const& memberName, const void* object, void* outData) {
  // pretty damn similar to the above
}

void ClassInfo::AddMember(string memberName, unique_ptr<IMemberInfo> member) {
  _members.insert(make_pair(move(memberName), move(member)));
}

void ClassInfo::AddBase(ClassInfo* classInfo, ptrdiff_t offset) {
  BaseInfo base;
  base.classInfo = classInfo;
  base.offset = offset;
  _bases.push_back(base);
}
That's the gist. You can add some templates to make it easy to add new entries, something like the following (or any of a bazillion possible variations):

void initializeReflection() {
  reflect_class(MyClass)
    .add_base<MyBase>()
    .add_member("position", &MyBase::_position)
  ;
}
You can also add other data like the names of the classes, names of the members, specific owner of each member, lots of error-checking, explicit serialization support, etc.

You can recurse through the types so that you can have a member variable be another reflected class (so you can have a class that has members of Vector3f) or even reflected members of unique_ptr or shared_ptr or container wrappers around other primitive or reflected types. It can get pretty intricate but you can support rather complex types with full I/O serialization as well as property GUIs.

#11 fastcall22   Crossbones+   -  Reputation: 4610

Like
2Likes
Like

Posted 14 July 2014 - 03:45 PM

Actually, if you're using any of the latest Visual Studio versions, there should be a local database file with your project that contains metadata about your project. Usually, this database is used for intellisense, but you can connect to the database externally and perform SQL queries against it. With this, you can query information about your project, classes, and its members. Extending this, you could write a PowerShell script to automatically generate reflection information from all of your classes. Then you could add this to your build process, and... well, I'm getting ahead of myself, here. smile.png
c3RhdGljIGNoYXIgeW91cl9tb21bMVVMTCA8PCA2NF07CnNwcmludGYoeW91cl9tb20sICJpcyBmYXQiKTs=

#12 Hodgman   Moderators   -  Reputation: 32893

Like
2Likes
Like

Posted 14 July 2014 - 03:45 PM

Check out my binding system if you like, here: https://code.google.com/p/eight/source/browse/include/eight/core/bind.h
It does a bit more than you need, but does calculate the sizes/offsets of the member variables.



#13 Pink Horror   Members   -  Reputation: 1285

Like
1Likes
Like

Posted 14 July 2014 - 09:22 PM

const ComponentDeclaration::AttributeVector vAttributes =
{
{ AttributeType::FLOAT, "x" },
{ AttributeType::FLOAT, "y" },
{ AttributeType::FLOAT, "z" },
}


I came on here to mention offsetof, though I see that you've found that. Instead of trying to get fancier, I would start out by implementing your original idea. Maybe it'll be a little annoying to type class names over and over, but it'll get something working that you can refactor:

const ComponentDeclaration::AttributeVector vAttributes =
{
{ AttributeType::FLOAT, "x", offsetof(Component, vPosition) + offsetof(Vector2f, x) },
{ AttributeType::FLOAT, "y", offsetof(Component, vPosition) + offsetof(Vector2f, y) },
{ AttributeType::FLOAT, "z", offsetof(Component, vPosition) + offsetof(Vector2f, z) },
}


#14 Juliean   GDNet+   -  Reputation: 2867

Like
0Likes
Like

Posted 15 July 2014 - 07:28 AM


I came on here to mention offsetof, though I see that you've found that. Instead of trying to get fancier, I would start out by implementing your original idea. Maybe it'll be a little annoying to type class names over and over, but it'll get something working that you can refactor:

 

Thats what I'm doing at the moment. Refactoring is quite a pain since the amount of components for my 2D + 3D stuff is quite high, though I think I'll even merge some components here and there... anyway, it works out quite well that way for now, and got me rid of ~1500 LOC, mostly in gui/editor-related stuff, which is about 10% of my 2D-plugins. I'm glad I have a 2D project to test first & refactor based on that though, because there is just way more stuff going on in the 3D departement, so it would take way longer to get things even to work, not talking about refactoring...

 

I also wanted to mention, for anyone who might stumble up to this, that you don't have to perform offsetof twice for nested classes, you can just say

{ AttributeType::FLOAT, "x", offsetof(Component, vPosition.x) },

which is actually quite nice.

 

@SeanMiddleditch:

 

Thats actually interesting, I'm not sure if its not even a bit complicated for what I'm trying to do, since I'll never have such things as multiple interheritance etc... Everything that I expose via those "interfaces" is going to be POD, too, at least for the moment. I'll keep it in mind though, I might even want to use this to improve the type system of my visual scripting system, while I'm at it, I think it would fit there quite well.

 


Actually, if you're using any of the latest Visual Studio versions, there should be a local database file with your project that contains metadata about your project. Usually, this database is used for intellisense, but you can connect to the database externally and perform SQL queries against it. With this, you can query information about your project, classes, and its members. Extending this, you could write a PowerShell script to automatically generate reflection information from all of your classes. Then you could add this to your build process, and... well, I'm getting ahead of myself, here. smile.png

 

Uh, I really wouldn't want to rely too heavily on Visual Studio for stuff like that. The plugins should work independately, which could be achieved by generating this reflection information to a personal file format... but I quess thats a bit overkill, though its neat to see whats possible :D I also kind of like the idea that it would work fully automatically, without having to write a single line of code for each component.

 


Check out my binding system if you like, here: https://code.google.com/p/eight/source/browse/include/eight/core/bind.h
It does a bit more than you need, but does calculate the sizes/offsets of the member variables.

 

I had a look, seems you are also using offsetof, which makes me a little more secure that this is at least a viable choice.

 

Thanks again for all the input, much appreciated.






Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS