Jump to content

  • Log In with Google      Sign In   
  • Create Account


Dll plugin system: cross plugin "shared" classes


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
17 replies to this topic

#1 Juliean   GDNet+   -  Reputation: 2453

Like
0Likes
Like

Posted 14 September 2013 - 01:59 PM

Hello,

 

I'm making good progress with my game engines dll plugin system. Basically, nothing but one constructor function needed to be exported by a plugin to make it work, up until now. While seperating my modules into seperate dll projects, I've come across one issue again I've ignored earlier. Now its really turning out to be a big problem. Take a look at a sample code from the shadow plugin:

	        auto vEntities = m_pEntities->EntitiesWithComponents<ShadowCaster>();

		size_t count = 0;
	        for(auto pEntity : vEntities)
	        {
                                // do other stuff here ...

				if(auto pLight = pEntity->GetComponent<Light>())
				{
					if(!pLight->pModel)
						continue;

					if(auto pMaterial = pLight->pModel->GetMaterial(2))
						pMaterial->SetTexture(3, m_pTextures->Get(L"ShadowBlur" + conv::ToString(count)));
				}
                }

Now there is the line with pEntity-GetComponent<Light>(), which comes from the deferred light plugin. Up until now they've all be in the same project as my engine, so it wasn't a problem. Now that I'm completely seperating them, I've got to come up with a solution. Internally the shadow plugin will require the deferred light plugin to being loaded, so I've basically got three choices I can think of:

 

- Implicitely link to the light dll in the shadow dll project, export the Light-component class from the dll and use it directly there. My least favourite option, as is produces a quite unclean mess (exporting classes with possible storages of e.g. std::wstring, which I'm using excessively to exchange data in my components, etc...), and requires at least all header files of the first plugin to create another plugin that depends upon it. Not even talking about all the other issue like binary compatibility, etc...

 

- Have one header file like "Dev.h" for each plugin that possibly supports extentsion, where one/multiple pure-virtual classes are declared. The component and other classes being used will then derive both the engines component base class and this interface. The interface will have specific getter/setter methods for all the components attributes, as well as a method for retrieving the internal choosen id to identify the type of component, when querying a entity e.g. . This approach will still require one header file from the first plugin to create any extention to it. Plus, it will require some unsafe casting, since from the entity I can only return a reference/pointer to the internal component base class, which I'll then have to cast to a pointer to the plugins component interface. I hope that doesn't sound too confusing, thinking about it I'm not even 100% sure if this would work. Plus, it will introduce quite an amount of additional work for developers of a plugin to support extentions, and possibly generate a lot of plugins that aren't extendable because they lack such a "dev" header file.

 

- Last option, I'll put a special "dll-component" interface (pure virtual class) in the engine. It will have one generic getter and setter, which takes a string as identifier and a void* to some data, and another method for retrieving mentioned id.  For standard components like developed inside the engine, this getters and setters are overridden with an emtpy implementation in the base component class. Only in the plugin the dev will need to implement it using e.g. a switch statement for the identifier string to map the hard-coded member attributes like name, size, etc... I'd then have each dll/plugin/module register their components id with a string, so that it can then later easily be accessed. The finished product would look like this:

auto vEntities = m_pEntities->EntitiesWithComponents<ShadowCaster>();

size_t count = 0;
for(auto pEntity : vEntities)
{
    // do other stuff here ...

    if(IComponentDll* pLight = pEntity->GetComponent(ComponentRegistry::GetId(L"Light")))
    {
        gfx::IModel* pModel = dynamic_cast<gfx::IModel*>(pLight->GetAttribute(L"Model")); // maybe some helper-function like GetComponentAttribute<gfx::IModel*>(pLight, L"Model");
        if(!pModel)
            continue;

        if(auto pMaterial = pModel->GetMaterial(2))
            pMaterial->SetTexture(3, m_pTextures->Get(L"ShadowBlur" + conv::ToString(count)));
    }
}

Now this does still appear a little flawed to me. Mostly because it still puts some work to the developer (implementing out the setter/getter), and introduces eigther blind-trust unsafe static or dynamic casting (I don't really like dynamic casting that much). But at least I wouldn't have to export more code, and the interface part would be generic on the engines side.

 

_____________________________

 

So, what do you say? Which one of those methods, if any, would you use? If none of them appears good to you, what would you do different? All suggestions are welcomed!


Edited by Juliean, 14 September 2013 - 02:03 PM.


Sponsor:

#2 dougbinks   Members   -  Reputation: 484

Like
1Likes
Like

Posted 15 September 2013 - 06:22 AM

I can't answer your question specifically, but a few ideas here might help:

 

A standard way of getting around dynamic casts is using a GetInterface method, passing in an interface id. That id can be declared in the interface header file as a static member of the interface structure/class in such a way that you can then use template methods. You can see something similar in the GetInterface members in this file.

 

 

As an aside, for performance I'd not use strings but a string hash or id. You can easily convert by adding a function to get the hash/id, use this outside the loop and then use hash/ids inside.

 

One other note is that if you're iterating through entities getting a component you should likely consider having that component stored in it's own array and then iterating through that.



#3 Juliean   GDNet+   -  Reputation: 2453

Like
0Likes
Like

Posted 15 September 2013 - 07:04 AM


A standard way of getting around dynamic casts is using a GetInterface method, passing in an interface id. That id can be declared in the interface header file as a static member of the interface structure/class in such a way that you can then use template methods. You can see something similar in the GetInterface members in this file.

 

Thanks, that sounds intersting indeed, but does it really help in the specific case here? The dynamic case comes from aquiring an attribute via the GetAttribute-method, which might internally look like this:

void* GetAttribute(const std::wstring& stName)
{
       // hashing migh be a good idea after all :D
       if(stName == L"Model")
                return m_pModel; // gfx::IModel* m_pModel;
       else if(stName == L"Name")
                return m_stName; // std::wstring m_stName;
       else
                return nullptr;
}

The problem here is more or less that I need to access an attribute of variable type, from int over std to custom interface. Can such a GetInterface-or similar method be applied here to eliminate this problem, or do I need another solution?

 


As an aside, for performance I'd not use strings but a string hash or id. You can easily convert by adding a function to get the hash/id, use this outside the loop and then use hash/ids inside.

 

Thanks, I'm planning on doing so, applying it overall will take some time though, and Nr1 priority so far is to seperate and get to work all modules as dll plugin...

 


One other note is that if you're iterating through entities getting a component you should likely consider having that component stored in it's own array and then iterating through that.

 

I'm not so sure about this. For example, here I need to access the light-component too, while originally iterating over the shadow components. I know that some people define it that the components are not equivalent to the entitiy, but things have worked well so far, and from what I get there are a hundred different views on entity/component systems. Still, I agree that some kind of sorting would be helpful, I'll be implementing some sort of cache-system to store buckets of entities based on their components. Unless I run into performance issue, where I'll consider applying a more cache-friendly system, but so far, even thousands of entities with tons of different systems sometimes cross-referencing components haven't had any extraordinary negative performance impact...



#4 dougbinks   Members   -  Reputation: 484

Like
0Likes
Like

Posted 15 September 2013 - 07:33 AM


The problem here is more or less that I need to access an attribute of variable type, from int over std to custom interface. Can such a GetInterface-or similar method be applied here to eliminate this problem, or do I need another solution?

 

Yes. Rather than return a void* you return an IInterface* type which has the virtual function for GetInterface defined. This then allows dynamic-cast free casting, made easier via templates. It still costs virtual function call though.

 


I'm not so sure about this. For example, here I need to access the light-component too, while originally iterating over the shadow components.

 

You can still access one component whilst iterating through another; you can get the entity for the light and then get the shadow component for that. Using arrays of each component also allows low level systems like renderers to completely avoid using any typing or entity system.



#5 Juliean   GDNet+   -  Reputation: 2453

Like
0Likes
Like

Posted 15 September 2013 - 09:18 AM


Yes. Rather than return a void* you return an IInterface* type which has the virtual function for GetInterface defined. This then allows dynamic-cast free casting, made easier via templates. It still costs virtual function call though.

 

Ah, so I'd have to create an IInterface derived class for every type I want to convert to, is this correct? Doesn't that lead to having to distribute the interfaces I want to convert to the end user after all? I think I'm missing some key piece, but wouldn't that then have to be used something like this:

IInterface* pModelInterface = pLight->GetAttribute(L"Model");
gfx::IModel* pModel = pModelInterface->GetInterface<IModelInterface>();
IInterface* pStringInterface = pLight->GetAttribute(L"Name");
const std::wstring& stName = pModelInterface->GetInterface<IStringInterface>();

Does that mean I have to eigther define the interfaces for all common types in the engine as part of the dll interface, or have the interfaces included by the plugins header file (which I'd want to avoid if possible); or am I really missing something? (some explanatory pseudo-code would be nice, in that case though smile.png )

 


You can still access one component whilst iterating through another; you can get the entity for the light and then get the shadow component for that. Using arrays of each component also allows low level systems like renderers to completely avoid using any typing or entity system.

 

I quess that would work, so I'll consider it a close option in case it is necessary, premature optimization and whatenot (or, lets add, me being to lazy to rework this part yet). As for low level systems, that certainly can be a nice addition, but I've seperated my low level render systems so that they are controlled via a gfx-layer, that is used by the component system, so that this wasn't really an advantage here, so there is no need/function for using anything from the entity system directly anyway ^^


Edited by Juliean, 15 September 2013 - 09:21 AM.


#6 dougbinks   Members   -  Reputation: 484

Like
1Likes
Like

Posted 15 September 2013 - 10:19 AM


Ah, so I'd have to create an IInterface derived class for every type I want to convert to, is this correct? Doesn't that lead to having to distribute the interfaces I want to convert to the end user after all?

 

If you want some form of fast type conversion without dynamic cast you'll need an interface type which you derive from, yes. In order for a plugin writer to be able to use code, they need the declaration of that code at minimum - there's little use being able to cast a type to another type if they can't then use that type. So yes you'll need to distribute interfaces for the functionality. An alternative is to write a message passing or signal/slot type system, but this still requires that you define some way for programmers to understand how to use this.

 


Does that mean I have to eigther define the interfaces for all common types in the engine as part of the dll interface, or have the interfaces included by the plugins header file (which I'd want to avoid if possible); or am I really missing something? (some explanatory pseudo-code would be nice, in that case though )

 

Not quite sure I'm getting the issue here. If you want someone to be able to use an interface they need the declaration of that interface, which is usually in a header file.

 


I quess that would work, so I'll consider it a close option in case it is necessary, premature optimization and whatenot (or, lets add, me being to lazy to rework this part yet). As for low level systems, that certainly can be a nice addition, but I've seperated my low level render systems so that they are controlled via a gfx-layer, that is used by the component system, so that this wasn't really an advantage here, so there is no need/function for using anything from the entity system directly anyway ^^

 

It's worth your while following your current course through to completion if you've already got some way in. The issue I'm referring to is not really a premature optimization, but just a good point to start. There's some good material to be found by searching the web for information on data oriented design (often referred to as DOD). It's also a very good principal to not overly generalize systems until you need to - there's some good information on that in this blog and comment discussion.



#7 Juliean   GDNet+   -  Reputation: 2453

Like
0Likes
Like

Posted 15 September 2013 - 11:41 AM


Not quite sure I'm getting the issue here. If you want someone to be able to use an interface they need the declaration of that interface, which is usually in a header file.

 

Well, I want to do the somehow direct opposite, namely use a component without having given the declaration of any specific interface of the plugin that. All the needed interfaces should be given by the engine itself - and it almost is. The BaseComponent-class is part of the engine, and every component is supposed to derive from it. So all that I need now is a somewhat safe method to retrieve specific attributes - if I was to rely on the dynamic_cast, there wouldn't be the need to have any header file for using another plugin, which is my final goal - the problem with requiring a header file is that the developer of the plugin will have to make sure that the plugin can be extendet explicetly - I'd prefer a method that puts most of the work at the engine, with having a clean interface that "simply" has to be implemented in order to allow anyone to develope their plugin on top of the engine and any other plugin, by just knowing what components it declared, and which data members it has. Just to emphasise, this is not about using entirely new classes/interfaces that the plugin defines - its just about working with the eigther POD or engine defined interface data members of the components, as sort of a messaging system.

 

I hope this helped to clarify a bit, and didn't go toward confusing you about my goals even more (I've got a real hard time expressing this, probably due to my limitations in the english language...). Does it appear more clear now?

 


The issue I'm referring to is not really a premature optimization, but just a good point to start.

 

My bad, I wasn't referring to your advice as premature optimization, but to the act of applying them for the reason I mentioned, as in cache-friendlyness, since other than that I'm pleased with the system as it (not meaning that I don't see any possibilty, or don't want to improve, but not right now).

 


It's also a very good principal to not overly generalize systems until you need to - there's some good information on that in this blog and comment discussion.

 

I do agree in matter of productivity especially in buisness, as in the article, but for this project, its not really any matter for me. Up until the next university semester starts, I'm pretty much coding the engine without any clear project, so up until then I just implement features I've got on my todo-list that will make further developement probably easier. Pretty much the whole game engine up until now was unnecessary for the one simple tower defense game I've developed until now, its a whole "future-paced" project if you will. Even if it would never be used in any other game project, unlike the future focused guy in the article, I'd still have enjoyed the time I spent developing, and it has kept me focused on the project for almost a year now without any major break, while normally I'm loosing interesting in hobby projects after a few weeks/months. Not that I don't appreciate the additional input, just to state the I'm well aware what I'm doing and why I'm doing it ;)



#8 AllEightUp   Moderators   -  Reputation: 4195

Like
1Likes
Like

Posted 15 September 2013 - 12:22 PM

Ignoring all the work up on plugin structure, it seems you are focused on generic attribute access. Focusing just on that feature, then the way I've always looked at this is a getter/setter solution either based on variants or by templates which do all the type checking. Using variants is the easiest solution though it requires a bit of dynamic memory for some types. Basically the variant return is:

struct Variant
{
  enum
  {
     kSInt, kUInt, kFloat, kDouble, etc...
  };
  uint32_t    Type;
  union
  {
    int32_t   SInt;
    uint32_t  UInt;
    etc
  };    
};
Now, you can retrieve the attributes using a "bool GetAttr( const std::string& attr, Variant& outVar )" generic interface. The bool result tells you if the attribute exists and then when you cast the variant to some type the accessors (the struct would be all private with cast helpers) would validate that the internal data casts to the type you request.

Using the same concept though, you can embed the variant like checks into the generic getter such as: "bool GetAttr( const std::string& attr, uint32_t typeKey, void* outData )". Again, you have the bool result saying the attribute exists or not but now you pass the type into the source and check the desired type is valid for the attribute in question. Internally you simple implement something like:

switch( typeKey )
{
  case Type::SInt:
    if( attr=="SomeVal" )
      *(reinterpret_cast< int32_t* >( outData )) = mSomeVal;
      return true;
It gives some small amount of type safety though of course people can mismatch the type key and the output value and cause all sorts of problem. You can also turn the entire thing into a table based solution such that you would write out:

  { Type::SInt, "SomeVal", offsetof( SomeClass, mSomeVal ) }
Then on startup build a nice hash table of the different types which should be about as fast as the switch/case stuff without as many likely typo's involved.

Finally, with the combined solution you can write templates such as:

template< typename _Type >
bool GetAttr( const std::string& name, _Type& outData )
{
  assert( false ); // Non-specialized type..  Use generic access.
  return false;
}

template<>
void GetAttr( const std::string& name, uint32_t& outData )
{
  return GetAttr( name, Type::UInt, &outData );
}
Overall the variant is the safest variation and best C++ style in the general sense but it comes at a performance/memory bandwidth cost compared to the less safe more "C" style accessor.


Now, with all the above, there is one other item to keep in mind when moving to the DLL solution. Memory allocations. I assume you either know about this or have worked around it. Basically unless all the plugins and the executable share a common copy of msvcrt<xxx>.dll, the memory will come from different memory managers and passing the memory around is unsafe. Even using the std::string stuff is unsafe since the memory comes from different allocators. In the past I simply wrote a dll which I called "Core" and linked that against the runtimes, then my executable and all plugins had to link against that to get the common allocator. It is not the best solution by far since then everyone has to use the same compiler, or at least the same version of msvcrt, but on the other hand, linking up shared memory management on Windows is a pure hell to get correct. Otherwise, you need to make absolutely sure you never pass memory back and forth between plugins without a dedicated "alloc/free" pair of functions in each plugin. I.e. even the most simple: "GetName()" would need an equivalent "FreeName( p )" without some solution to the memory problem.

Just another gotcha to worry about.

#9 dougbinks   Members   -  Reputation: 484

Like
0Likes
Like

Posted 15 September 2013 - 02:07 PM


Well, I want to do the somehow direct opposite, namely use a component without having given the declaration of any specific interface of the plugin that. All the needed interfaces should be given by the engine itself - and it almost is. The BaseComponent-class is part of the engine, and every component is supposed to derive from it. So all that I need now is a somewhat safe method to retrieve specific attributes - if I was to rely on the dynamic_cast, there wouldn't be the need to have any header file for using another plugin, which is my final goal

 

You cannot dynamic_cast to a type which has not been fully defined, so the limitations are the same as for the GetInterface system. Note that you each plugin does not need to have it's own interface, they can implement interfaces you have defined in your engine.



#10 Juliean   GDNet+   -  Reputation: 2453

Like
0Likes
Like

Posted 15 September 2013 - 04:11 PM

@AllEightUp:

 

Hm, that sounds like it, but also complicated again, having to define lots of stuff for each possible data type, which can get quite enormous (theoretically, I'd have to define each primitive type, engine class and possible common std::classes). I quess the safety comes at a cost, but I didn't expect that it would need so much extra work per type... I thought there'd be a more general solution. Well, browsing through the web I've found kind of a universal solution, still requiring RTTI but appearantly way faster, which I could apply in a helper function. I'm talking about using typeid. Since I only care about that the types are equal, wouldn't it fare just as well if I just did that?

class BaseComponent
{
public:
	template<typename Type>
	Type* GetAttributeSafe(const std::wstring& stName)
	{
		std::type_info inType;
		if(auto pAttribute = GetAttribute(stName, &inType))
		{
			if(inType == typeid(Type))
				return (Type*)pAttribute;
		}

		return nullptr;
	}

protected:

	virtual void* GetAttribute(const std::wstring& stName, std::type_info& outType) const = 0;
}

class SpecificComponent: BaseComponent
{
private:
	void* GetAttribute(const std::wstring& stName, std::type_info& outType)
	{
		if(stName == L"Model")
		{
			 outType = typeid(m_pModel);
			 return m_pModel;
		}
		
		return nullptr;
	}

	gfx::IModel* m_pModel;
}

// usage:
gfx::IModel* pModel = pLight->GetAttribute<gfx::IModel>(L"Model"); // returns nullptr if eigther attribute doesn't exist or types doesn't match

Thats kind of like what you suggested, just with using RTTI instead of making up a custom system, and thus not having to do a custom . Does that sound reasonable, or are there any drawbacks I'm not seeing right now?

 

 

 


Now, with all the above, there is one other item to keep in mind when moving to the DLL solution. Memory allocations. I assume you either know about this or have worked around it. Basically unless all the plugins and the executable share a common copy of msvcrt.dll, the memory will come from different memory managers and passing the memory around is unsafe. Even using the std::string stuff is unsafe since the memory comes from different allocators. In the past I simply wrote a dll which I called "Core" and linked that against the runtimes, then my executable and all plugins had to link against that to get the common allocator. It is not the best solution by far since then everyone has to use the same compiler, or at least the same version of msvcrt, but on the other hand, linking up shared memory management on Windows is a pure hell to get correct. Otherwise, you need to make absolutely sure you never pass memory back and forth between plugins without a dedicated "alloc/free" pair of functions in each plugin. I.e. even the most simple: "GetName()" would need an equivalent "FreeName( p )" without some solution to the memory problem.

 

I've definately heard of it, thats why I tried to keep as much as possible internal in my plugins. But what I've also heard so far is that using pointer/references is somewhat safe, like that a std::wstring& will work cross-compiler, since its a pointer and the size will be the same regardless, while only the internal "layout" of the wstring will differ... am I missinformed? Plus, right now my whole engine is a dll that both the plugins and the application have to link against, that seems to be just what you are talking about, isn't it? By god, I don't want to write a dealloc-function for every simply getter I export :/

 

 

 


You cannot dynamic_cast to a type which has not been fully defined, so the limitations are the same as for the GetInterface system. Note that you each plugin does not need to have it's own interface, they can implement interfaces you have defined in your engine.

 

Hmm, thats not really what I wanted to imply, the only real "problem" I had with the GetInterface solution (as well as AllEightUps) was the I required to declare a new interface with custom id etc.. for each type that was possibly going to be used, which at least in my had made it seem a little much work, where I'd prefered a more generalized method. In that matter, what are your thoughts on the method using typeid I proposed before?

 

Oh, and refering back to your comment about DOD, I've actually just now realized that my low level render system is in fact data driven, I'm employing a render queue system based on render commands processed in such a linear fassion as mentioned in the presentation, quess I'm not to far offtrack. Just as a quick note. Its strange to see that you are already using a "pattern" without really knowing its name xD


Edited by Juliean, 15 September 2013 - 04:32 PM.


#11 AllEightUp   Moderators   -  Reputation: 4195

Like
1Likes
Like

Posted 15 September 2013 - 09:57 PM

Hm, that sounds like it, but also complicated again, having to define lots of stuff for each possible data type, which can get quite enormous (theoretically, I'd have to define each primitive type, engine class and possible common std::classes). I quess the safety comes at a cost, but I didn't expect that it would need so much extra work per type... I thought there'd be a more general solution. Well, browsing through the web I've found kind of a universal solution, still requiring RTTI but appearantly way faster, which I could apply in a helper function. I'm talking about using typeid. Since I only care about that the types are equal, wouldn't it fare just as well if I just did that?

Thats kind of like what you suggested, just with using RTTI instead of making up a custom system, and thus not having to do a custom . Does that sound reasonable, or are there any drawbacks I'm not seeing right now?


Unfortunately there is a gotcha in that. If I'm not mistaken, typeid returns a const type_info reference which the content of can (will?) change between dll's. With so's on *nix variants, it is probably all nice and safe given the consistent memory model, that is not the case on Windows and I'm almost positive this would end up with multiple type_info's which don't match if passed over dll boundaries. This and the bloat/performance issues with RTTI is why you see so many games that roll their own RTTI replacements, and it is almost always only a problem on Window's because of the memory model. I haven't investigated it myself in a long time, so all I can suggest is testing it in debug/release modes and being very careful before you decide it works.

Now, with all the above, there is one other item to keep in mind when moving to the DLL solution. Memory allocations. I assume you either know about this or have worked around it. Basically unless all the plugins and the executable share a common copy of msvcrt.dll, the memory will come from different memory managers and passing the memory around is unsafe. Even using the std::string stuff is unsafe since the memory comes from different allocators. In the past I
<snippy>

 
I've definately heard of it, thats why I tried to keep as much as possible internal in my plugins. But what I've also heard so far is that using pointer/references is somewhat safe, like that a std::wstring& will work cross-compiler, since its a pointer and the size will be the same regardless, while only the internal "layout" of the wstring will differ... am I missinformed? Plus, right now my whole engine is a dll that both the plugins and the application have to link against, that seems to be just what you are talking about, isn't it? By god, I don't want to write a dealloc-function for every simply getter I export :/

Unfortunately things are considerably trickier.. sad.png Yes, reference counted objects will solve this and by nature you might think that stl takes care of the problem, you'd be wrong and bad things are going to happen. Take the silly example:

// in your plugin:
const std::string  GetName() const;  // Note: non-reference for example..

// in other plugin or somewhere:
const std::string theName = thePlugin->GetName();
When 'theName' leaves scope you are going to trash the allocator list in the calling code. The reference count works as intended, the memory transfers to "theName" as intended, the reference is increased, the temporary goes away and the reference is back to 1. (or a move happens that just puts it directly in theName, either way) Unfortunately the problem is that the allocation took place in the memory space of the plugin and because stl is header only and likely inlined the delete call takes place in the other plugin or program which made the call. Boom, you just leaked in the plugin and corrupted the callers memory.

You have a number of ways to correct such things:

1. Guarantee *all* plugins, executables etc link to the dll version of msvcrt.
2. Make a dll library which everything links to which provides the link the msvcrt. (Actually a bit safer than #1 because it guarantee's everyone uses the same version.)
3. Use custom allocators for all STL. The functions must be behind a non-inlinable interface and in a single location. This is a pain in the butt and I don't suggest trying it, too easy to miss things.

Probably more solutions. Sorry to make it sound like such a pain. #1 or #2 are not particularly difficult.

#12 dougbinks   Members   -  Reputation: 484

Like
1Likes
Like

Posted 16 September 2013 - 05:25 AM


Hmm, thats not really what I wanted to imply, the only real "problem" I had with the GetInterface solution (as well as AllEightUps) was the I required to declare a new interface with custom id etc.. for each type that was possibly going to be used, which at least in my had made it seem a little much work, where I'd prefered a more generalized method. In that matter, what are your thoughts on the method using typeid I proposed before?

 

In your case typeid would likely work as well as a dynamic cast - both end up calling strcmp in order to get cross dll compatibility. For this reason generally all performance aware applications like games generally end up writing their own type system like I've described, and additionally many games build without RTTI.

 


Guarantee *all* plugins, executables etc link to the dll version of msvcrt.

 

Note that it has to be the exact same version - the debug version is not compatible with the release version.



#13 Juliean   GDNet+   -  Reputation: 2453

Like
0Likes
Like

Posted 16 September 2013 - 06:59 AM


Unfortunately there is a gotcha in that. If I'm not mistaken, typeid returns a const type_info reference which the content of can (will?) change between dll's. With so's on *nix variants, it is probably all nice and safe given the consistent memory model, that is not the case on Windows and I'm almost positive this would end up with multiple type_info's which don't match if passed over dll boundaries. This and the bloat/performance issues with RTTI is why you see so many games that roll their own RTTI replacements, and it is almost always only a problem on Window's because of the memory model. I haven't investigated it myself in a long time, so all I can suggest is testing it in debug/release modes and being very careful before you decide it works.

 

As far as I've tested it in debug mode, the references fromt typeid are equal and will compare true even cross-dll. I'll have to configure plugins for release mode first, so I'll test it at another timer, but so far, it at least seems to work. I'll do some more in-depth testing but for now, just to get all my modules up and running as plugins, I'll consider it at least a temporary base.

 


You have a number of ways to correct such things:

1. Guarantee *all* plugins, executables etc link to the dll version of msvcrt.
2. Make a dll library which everything links to which provides the link the msvcrt. (Actually a bit safer than #1 because it guarantee's everyone uses the same version.)
3. Use custom allocators for all STL. The functions must be behind a non-inlinable interface and in a single location. This is a pain in the butt and I don't suggest trying it, too easy to miss things.

Probably more solutions. Sorry to make it sound like such a pain. #1 or #2 are not particularly difficult.

 

Uh wow, thats much more complicated that I thought it'd be. Now granted, I wanted to wait with implementing dll plugins some more time, but after I've relealized its at least doable (though I'm pretty sure I'm far away from portability or anything like it), I wanted to do go for it right away. At least I'm learning a lot of new things. Regarding the linking to msvcrt, do I have to manually make sure that the dll libary everything likes to (as in solution #2) is linking to a specific version of msvcrt? If so, how would I achieve that?

 

 

 


In your case typeid would likely work as well as a dynamic cast - both end up calling strcmp in order to get cross dll compatibility. For this reason generally all performance aware applications like games generally end up writing their own type system like I've described, and additionally many games build without RTTI.

 

Regarding the strcmp, I'm not so sure.. at least in debug build, even the returned hash_code() is equal cross-dll, so I suppose it could do that instead, making it at least a bit faster than dynamic_cast, also due to that those are compiler constants and dynamic_cast has to traverse some more... I'm not so sure, just from what I've read in numerous posts on stackoverflow, etc... .Thinking about it, I wonder if dynamic_cast would even work in any way? I mean, for the virtual overloaded GetAttribute()-method, I can just return void*, since I can't make it a template, since this can't be made virtual... and I've heard dynamic_cast won't work with void*. So just out of interest, I'm not employing it anyway, but would there be any way to get dynamic cast to work here (see my example code in my last post before this one)?

 

Regarding the performance issue, I do get what you are saying, and its definately noted on my todo list, eigther if the typid-thing doesn't work e.g. in release mode, or if I'm noticing performance issues here, or just as a general optimization I'll apply when I feel a lack of motivation to implement new features or so. In that regard, thanks to you and AllEightUp for offering me those solutions, I'll have to decide which one I'm going to apply though when it come to it.

 

Just one note, is there any possible way in any of you guys methods to at least automatize the way of "registering" the types at compile time? Something like

// in one header file:

REGISTER_TYPE<int>();
REGISTER_TYPE<float>();
REGISTER_TYPE<gfx::IModel>();
REGISTER_TYPE<std:vector<int>();

That somehow auto-generates all the interfaces/entries and type ids?

 

EDIT:

 

Thinking more in-depth about it, can't I just go:

// in typeinfo.h

class TypeInfo
{
public:
	static size_t GenerateTypeId(void);
private:
	static size_t typeId;
};

// in typeinfo.cpp
size_t TypeInfo::GenerateTypeId(void)
{
	return typeId++;
}

// in type.h
template<typename Type>
size_t GetTypeId(void)
{
	static size_t type = TypeInfo::GenerateTypeId();
	return type;
}

template<typename Type, typename Type2>
bool TypesEqual(void)
{
	return GetTypeId<Type>() == GetTypeId<Type2>();
}

// in Component.h

template<typename Type>
Type* BaseComponent::GetAttributeSafe(const std::wstring& stName)
{
	size_t typeId = 0;
	if(auto pAttribute = GetAttribute(stName, &typeId))
	{
		if(typeId == GetTypeId<Type>())
			return (Type*)pAttribute;
	}

	return nullptr;
}

// implementatation

void* Light::GetAttribute(const std::wstring& stName, size_t& type) const
{
	if(stName == L"Model")
	{
		type = GetTypeId<gfx::IModel>(); // maybe add some ReturnAttributeSafe - template method or macro to unsure type and id match
		return pModel;
	}

	return nullptr;
}

// usage:

gfx::IModel* pModel = pLight->GetAttribute<gfx::IModel>(L"Model");

Wouldn't that work? It would both be fast (one virtual function call for the unavoidable overloaded GetAttribute-method) and secure to a certain point. It would not require any special work for a new type, and only types really used would be created. Or are there any more gotchas I'm missing? Any thoughts on it compared to the GetInterface/Variant method?


Edited by Juliean, 16 September 2013 - 07:42 AM.


#14 dougbinks   Members   -  Reputation: 484

Like
0Likes
Like

Posted 16 September 2013 - 08:45 AM


Wouldn't that work?

 

An automatic local id system is probably do-able, but there are gotchas. I think the example you use may not work as the template function may get instantiated multiple times (at least once for each dll) leading to multiple static members with different values for type for the same Type.



#15 AllEightUp   Moderators   -  Reputation: 4195

Like
0Likes
Like

Posted 16 September 2013 - 11:39 AM

Guarantee *all* plugins, executables etc link to the dll version of msvcrt.

 
Note that it has to be the exact same version - the debug version is not compatible with the release version.


Actually, if you have a fixed ABI such as pure virtual interfaces in front of the plugins and no cases of passed memory for the various reasons given, using a debug version or different msvc's is quite possible. Of course in such a case you are basically using a stripped down variation of COM which is what I have almost always used. COM as a "pattern", not the MS implementation, is actually quite nice and solid with many benefits such as this ability to mix and match so long as the ABI's are standardized.

#16 AllEightUp   Moderators   -  Reputation: 4195

Like
0Likes
Like

Posted 16 September 2013 - 11:51 AM

Wouldn't that work?

 
An automatic local id system is probably do-able, but there are gotchas. I think the example you use may not work as the template function may get instantiated multiple times (at least once for each dll) leading to multiple static members with different values for type for the same Type.


I suspect a runtime ID system will not be effectively available until VC gets it's act together and implements C++11 'constexpr'. I've done many experiments on this and the best solution I know of is generally a hash function with salt. (I.e. item name + say header where it is defined name.) Of course you have potential issues with collisions (can be caught in debug builds) but without constexpr you can't get it to be really general purpose. I wrote an experimental Clang implementation of djb2 hashing which happens at compile time and is independent of modules, but of course it only works with GCC/Clang since VC throws a tantrum right now. sad.png

#17 dougbinks   Members   -  Reputation: 484

Like
0Likes
Like

Posted 16 September 2013 - 12:07 PM


I suspect a runtime ID system will not be effectively available until VC gets it's act together and implements C++11 'constexpr'. I've done many experiments on this and the best solution I know of is generally a hash function with salt. (I.e. item name + say header where it is defined name.) Of course you have potential issues with collisions (can be caught in debug builds) but without constexpr you can't get it to be really general purpose. I wrote an experimental Clang implementation of djb2 hashing which happens at compile time and is independent of modules, but of course it only works with GCC/Clang since VC throws a tantrum right now.

 

The lack of constexpr is certainly a nuisance. You can do compile-time hashing with VS though - have a look here (and in the comments): http://www.altdevblogaday.com/2011/10/27/quasi-compile-time-string-hashing/



#18 AllEightUp   Moderators   -  Reputation: 4195

Like
0Likes
Like

Posted 16 September 2013 - 12:27 PM

I suspect a runtime ID system will not be effectively available until VC gets it's act together and implements C++11 'constexpr'. I've done many experiments on this and the best solution I know of is generally a hash function with salt. (I.e. item name + say header where it is defined name.) Of course you have potential issues with collisions (can be caught in debug builds) but without constexpr you can't get it to be really general purpose. I wrote an experimental Clang implementation of djb2 hashing which happens at compile time and is independent of modules, but of course it only works with GCC/Clang since VC throws a tantrum right now.

 
The lack of constexpr is certainly a nuisance. You can do compile-time hashing with VS though - have a look here (and in the comments): http://www.altdevblogaday.com/2011/10/27/quasi-compile-time-string-hashing/


Yup, the hash is possible at run time but the general purpose nature is missing. Some of the rather annoying limitations are that it can't be used in switch statements, constants in templates, enumeration assignments etc. Without the constexpr keyword I found such implementations to be mostly curiosity items, not to say they don't have some cases of usability, they unfortunately lack general application.




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS