# How to store data for reflection in preprocessor stage

This topic is 1057 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

Hey, I have just been studying how to implement reflection in the simplest and least expensive way possible in c++.

I've been studying up on macros, since those are all pre-processor, but I can't seem to find a way to accomplish what I am trying to do.

Sorry if this sounds kinda stupid. I am just sorta learning this stuff. I want to write a macro function that takes a string literal (for a variable or function name), and put it in a vector. The only problem is, I don't know how to create a vector in the preprocessing stage.

I was wondering if it is possible to create some sort of 'macro class' or struct to store this data, so that I can use this for later.

For example, I want to do something like this:

Psuedo-code:
#include <iostream>
#include <string>
#include <vector>

#define struct ReflectionData /
{/
std::vector<string> names;/
}

int main()
{
std::cout << ReflectionData.names[0];
std::cout << ReflectionData.names[1];
std::cout << ReflectionData.names[2];

return 0;
}

ADD_REFLECTION_DATA("Bowser")
Output:
Mario
Luigi
Bowser


Any idea how to accomplish something like this?

Edited by Solid_Spy

##### Share on other sites
Not sure what you're after but the preprocessor is totally unaware of C++ constructs like structs.
It runs before the compiler even starts munching through your code and acts as a text replacer.

##### Share on other sites

Not sure what you're after but the preprocessor is totally unaware of C++ constructs like structs.
It runs before the compiler even starts munching through your code and acts as a text replacer.

I want the program to store strings inside of a vector during the preprocessor stage, and then output the data after it has been compiled during runtime. I eventually want to also store boost::function_pointers to functions in the classes, and use the data for metaprogramming, so that other classes can call each others functions without circular dependencies, similar to what Unity and UE4 does.

Edited by Solid_Spy

##### Share on other sites

Ok, I still haven't figured out how to do it but I managed to get somewhere closer to the answer, and this article helped: http://www.randygaul.net/2012/10/01/c-reflection-type-metadata-introduction/

here is the code that I wrote:

#include <windows.h>

#include "reflector.h"

int main()
{
std::string reflectName = ReflectionManager::names[0];

std::cout << '\n' << reflectName;
return 0;
}


#ifndef REFLECTOR_H_
#define REFLECTOR_H_

#include <iostream>
#include <string>
#include <vector>

class ReflectionManager
{
public:
static std::vector< std::string > names;

{
names.push_back(name);
};
};

std::vector< std::string > ReflectionManager::names;

#endif /* REFLECTOR_H_ */


Unfortunately this doesn't work if I place the REFL_ADD_NAME("mario"); function outside of main, which leads me to believe that the macro function is being called at runtime. Ugh. I wan't the macro to be called only at the preprocessing stage. Any info on how to do this?

Edited by Solid_Spy

##### Share on other sites

You can not run code outside of a function (with the exception of putting it into a constructor for a static variable in C++). Thats basic C/C++ knowledge. Maybe you should first learn a language before shoehorning concepts of other languages into the already most complicated language or simply just use the other language when you need features from it?

##### Share on other sites
Ignoring what you're even trying to do, the preprocessor commands mostly just act as find and replace, they tend to either copy text from one place to another or do light processing of text before pasting it in(as is the case with macros.)

The preprocessor doesn't understand concepts like classes, it will never know what a vector is and honestly I don't know why you're attempting this when the code you've shown so far lacks a very basic understanding of what the code is actually doing at different stages. When anything is being "called" it is going to be during runtime.

##### Share on other sites

You can not run code outside of a function (with the exception of putting it into a constructor for a static variable in C++). Thats basic C/C++ knowledge.

*sigh, I understand you can't run code outside a function, that's why i'm asking for help. Afaik Boost::mpl has a good solution that runs preprocessor code outside of functions and during compile time using template metaprogramming. I might use that instead.

Edited by Solid_Spy

##### Share on other sites

I think you are approaching it the wrong way.

If you want things to happen at compile time, you need to only use constructs that are compile time.

Your problem isn't the macros, but what they expand into.

Any function calls, unless they are constexpr is out of the question, you simply can't get .push_back to work in compile time, not even if you wrap it in another function call or any number of macros or templates.

You need to find some other way to construct your reflection data map which is compile time only, if that is what you want.

##### Share on other sites
If you really want to understand macros, I suggest searching your compiler for a way to output the preprocessed cpp file so you can see how they expand.

In MSVC you can right click on a cpp file, select "Properties", expand "C/C++", select "Preprocess", and set "Preprocess to a File" to "Yes". Then just right-click on that one file and compile it - it won't actually build it into a program, but it will produce a text file on disk that you can then open in MSVC to see the output. (Remember to change the option back to "No" in order to actually build your program again!)

To make a long story short though as several people have said, macros are copy-paste. If you can't do what you're trying to do with Ctrl+C and Ctrl+V by hand, you can't do it in macros.

Addendum - Heavy macro use is generally frowned upon in C++, especially as templates have gotten more powerful in the later iterations of the language. They do not understand any C or C++ features and do not respect scoping rules. While there are still some things that only they can do (conditional compilation, for example), that list is rapidly shrinking. Edited by SmkViper

##### Share on other sites

I want the program to store strings inside of a vector during the preprocessor stage, and then output the data after it has been compiled during runtime. I eventually want to also store boost::function_pointers to functions in the classes, and use the data for metaprogramming, so that other classes can call each others functions without circular dependencies, similar to what Unity and UE4 does.

Uh, at least what Unreal does for their reflection involves a custom preprocessing code-generator. Their macros I belive only serve as input for their own preprocessing tool, which then generates the actual code in a seperate header file (called Header.h.generated in their case). So unfortunately I don't belive that what you want to do can be done without writing your own compiler/processing tools/routines.

Edited by Juliean

##### Share on other sites
Olof Hedman have the right idea. C++ can let you approach reflection in a unique way: do (almost) everything at compilation time.

I've experimented and played around a lot with that in a couple of pet projects as well as in some production code at my previous company. The solution I've ended up with that I liked the most is to describe everything at compilation time using template specialization, much like type traits. From various proposals for compilation time reflection for the C++ standard that I've seen, I think other people have used similar approaches to reflection in c++ as they are proposing things that work generally in a similar way.

I'm going to oversimplify this, but this approach is very extensible. I've used it to automatically generate language bindings completely at compilation time, to serialize graphs of objects, and a few other things.

Basically, I have a reflection::traits template that I specialize to describe every item I want my reflection system to describe. It is defined like so:

namespace reflection
{
template< typename T > struct traits {};
}


I then have a specialization of it for each thing I want to describe, and which contain static methods, typedefs, etc. depending on what kind of thing I'm describing.

For instance if I have a class named Foo, I'll have a specialization of the reflection template that looks like this:
template<> struct reflection::traits< Foo >
{
static const char* Name() { return "Foo"; }
};

At runtime, I now can now the name of class Foo by calling this: reflection::traits< Foo >::Name()

Of course, just getting the name isn't really useful. What I really want to do is to enumerate all the members. I do it using a compilation time visitor pattern. I guess it could be possible to use a more functional programming style, using tuples or something similar but I find the visitor approach less daunting.

In my previous company we only used this to serialize things, so I was doing something like to describe member variables:

template<> struct reflection::traits< Foo >
{
static const char* Name() { return "Foo"; }

template< typename V > accept( V& visitor )
{
visitor.template memberVar( "Blah", &Foo::m_Blah );
visitor.template memberVar( "Thing", &Foo::m_Thing );
}
};

It was all done using a bunch of macros so it looked something like this:

REFLECTION_START( Foo )
CLASS_MEMBER_VAR( Blah )
CLASS_MEMBER_VAR( Thing )
REFLECTION_END

The reflection::traits specialization had to be declared friend, so I had another macro for that. It is the only thing I need to insert into the definition of my classes, other than that this approach is non-intrusive, all the reflection stuff lives completely on the side.

It is possible to do much more complicated things, though: just create a dummy type for each member variable / property and create specific specializations of reflection::traits for those where you can then put whatever you need, like pointers to setter/getter member functions).

Likewise, macros are just one way to go about it. On my pet project I do more complicated things so I have a generator that generate all those templates from descriptions written in a simple language (I just don't like to insert annotations in the C++ code itself, I think it's both more complicated and less neat).

Then I can for instance print the member variables of any class described using this system by doing something like this:

template< class C >
class PrintObjectVisitor
{
public:
WriteVisitor( const C& object, ostream& output ) :
m_object( object ),
m_output( output )
{}

template< typename T > void memberVar( const char* pName, T C::* mpVar )
{
output << "  " << pName << ": " << m_object.*mpVar;
}

private:
const C& m_object;
ostream& m_output;
}

template< typename C >
void PrintObject( const C& obj )
{
PrintObjectVisitor v( obj, cout );
reflection::traits< C >::accept( v );
}

The visitor can have methods besides "memberVar" to describe other aspects of the class, using template functions to pass along the required type (so that the visitor can then use reflection on that type in the same way and so on). For instance, you could have a method to tell the visitor about the superclasses of the class. It can then recursively visit them to print their member variables too.

You can use this to attach reflection information and implement visitor patterns for other things than classes. For namespaces, I just create a dummmy type in the namespace:
namespace Bar
{
struct reflection_tag {}
}

Then specialize reflection::traits for "Bar::reflection_tag" to describe reflection informations about that namespace, including a function that goes through all the classes and nested namespace that it contains using a compile-time visitor like above.

Likewise, I create dummy structs in the class reflection traits to identify methods, properties and so on and then specialize the reflection::traits class for those to describe everything I need to describe about those things, depending on what I need to do.

The nice thing is that for most things, you pay no runtime cost for all that. That PrintObject function above, for instance, gets compiled into completely straight forward code that will just print each variable of the object without performing any lookup through a container at runtime. Furthermore, you don't get a bunch of data you don't need compiled into your binaries. If you only need to serialize a bunch of data in a binary blob, you don't need the class and variable names as long as you can identify the format version (I was doing it by using that same system to generate a CRC of all the classes description - it was enough for us since we used this only for network communication and it allowed to make sure that both the client and server were able to serialize and unserialize the exact same types). By the way in a modenr C++ compiler, things like computing CRCs such as this could be also done completely at compilation time using constexpr.

Another plus of this method is that you don't need to add stuff into the classes themselves, it lives completely on the side. You can make macros to build all those template specializations, I've did that at my prevoous company. However, in my personal projects I'm doing more sophisticated stuff using this approach than just serialization (like scripting language bindings), so I wrote a tool that generate those. I didn't want to use a Qt or unreal like approach of inserting special annotations through my c++ classes though, just a matter of taste but I find this messy. Instead, I have a simple interface description living in their own files, using a simple description language that ressembles a very simplified subset of c++, where I describe namespaces, classes, methods and properties. Then I have a tool that spits out a bunch of header files containing all those reflection::traits specialization, and from that I can generate  serialization, language bindings and such entirely through a bunch of templates.

Its also possible to use all this to construct a more classical system that allows introspection at runtime, but I'd only use that for things like property editor UIs and the like.
Edited by Zlodo

##### Share on other sites
Great post, Zlodo, thank you.

##### Share on other sites

I wouldn't bother doing a lot at compile time. I prefer to just load everything by explicitly calling functions. If you really don't like this for some reason you can instead use the constructor of an object at file scope to run data type registration code.

Example:

REGISTER_TYPE( int );

Would create some instance of a class right there at file scope. The constructor could be given a string "int", the sizeof( int ), and anything else you want. When the program starts up this information can be stored and used later.

##### Share on other sites

This topic is 1057 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.