Skip to main content
GameDev.net gamedev.net
🔒 Locked

Some General PlugIn Questions

Unity
Started by Markie Nov 30, 2009 at 1:42 PM 7 replies 1.3k views
Original Post
Markie
Markie
Hi all, I'm embarking on a life-long C++ journey (I guess) to create an architecture to simulate old-style pen & paper RPGS: The Multiple Role Playing Simulator As I need maximum modularity, I'm starting off with Plugins right from the start. I've read Petzold's brief chapter in "Programming Windows" and what Noel Llopis has to say in "C++ for Game Programmers". I've also read the following resources, but still have a few questions: - GameDev Forum: free c++ plugin system - Modular Programming with DLLs - Building a Better Plugin Architecture (C++) - Dynamic Plug-Ins for C++ - Building Your Own Plugin Framework Despite all the information found there, I still have some very basic, perhaps "newbie" type of questions: 1.) According to Petzold, Windows DLLs reside in some kind of common memory (like a part of Windows itself) which is not part of an application program's memory and which multiple applications can access. At the same time any data maintained by a DLL is different for each process:
Quote:
Multiple processes can share the same code in a dynamic-link library. However, the data maintained by a DLL is different for each process. Each process has its own address space for any data the DLL uses.
Now my question: Suppose I create a Plugin-system with an abstract base class (interface) for Game Objects residing in the main application and many derived subclasses which create the actual Game Objects. I put the derived Game Object subclasses into plugins (DLLs). Then I add to each plugin a factory function or a factory object which is "exported" to the main application and which, when called by the main application, creates the actual Game Object which is also in the plugin (but not exported "publicly"). Where is the Game Object actually created then? Is it created in the main program's memory, the heap? Or is it created in the plugin's process-specific memory? If the latter, is it a problem to create thousands of objects there (memory or speed constraints)? If we pass the created Game Object to the main program as a return-argument (NOT as a pointer!) of the factory function which created it, it's freed from the process-specific memory of the DLL as soon as the factory function terminates, right? Then the Game Object would be re-created in the memory of the main program (as passing by argument makes a copy), probably on the heap, right? If so, how would the main program know how much memory to reserve there for the Game Object if all it has is the implementation of the abstract base class (interface) of the Game Object, but not the implementation of the actual Game Object which it is passed? 2. Studying different sources about plugins I found that Windows seems to support C++ plugins, that is, one can directly put a class into a plugin without a problem. UNIX does not. In UNIX you MUST create a C++ class object in a plugin with a C-style function, right? The article above "Building a Better Plugin Architecture" suggest, instead of exporting a function from a plugin to create an object implemented in a plugin (and perhaps one to destroy it), it would be better to create a (distributed / ABC) factory and one derived factory subclass for each kind of object in a plugin. Then the main program can call this factory which creates and destroys the objects implemented in the same plugin. To create a platform-independent plugin system this means every plugin needs to contain the following parts, right? - The implementation of the derived object class which the plugin should add to the main application. - The implementation of the derived factory class which produces and destroys the above mentioned object. - Because UNIX cannot access C++ objects in plugins directly, we also need C-style function(s) which create and destroy the (singleton) factory object above. These are exported to the main program. Did I get all this right? To build a good plugin architecture as described in "Building a Better Plugin Architecture", every plugin needs an implementation of the object it wants to add to the main program, an implementation of the factory to create it AND C-style Create() - and Destroy() functions which create and destroy the factory object? Sorry if I'm not quite clear about all this. Any help is appreciated! :) Greetings, Mark
Sneftel
Sneftel
Quote:
Original post by Markie
1.)
Where is the Game Object actually created then? Is it created in the main program's memory, the heap? Or is it created in the plugin's process-specific memory? If the latter, is it a problem to create thousands of objects there (memory or speed constraints)?
DLLs share the application's memory space. It's the same process. As for allocation, see this and this.
Quote:
If we pass the created Game Object to the main program as a return-argument (NOT as a pointer!) of the factory function which created it, it's freed from the process-specific memory of the DLL as soon as the factory function terminates, right? Then the Game Object would be re-created in the memory of the main program (as passing by argument makes a copy), probably on the heap, right?
Sure, but of course that won't allow you to do virtual function binding, which makes it pretty much useless for plug-in stuff.
Quote:
2. Studying different sources about plugins I found that Windows seems to support C++ plugins, that is, one can directly put a class into a plugin without a problem. UNIX does not. In UNIX you MUST create a C++ class object in a plugin with a C-style function, right?
There is nothing in Windows or UNIX that specifically allows or prohibits exporting class interfaces from dynamic libraries. All major C++ compilers I know of allow this, although ABI compatibility is an issue.
Quote:
The article above "Building a Better Plugin Architecture" suggest, instead of exporting a function from a plugin to create an object implemented in a plugin (and perhaps one to destroy it), it would be better to create a (distributed / ABC) factory and one derived factory subclass for each kind of object in a plugin. Then the main program can call this factory which creates and destroys the objects implemented in the same plugin.
The first sentence in that article is "This article will guide you through the design of a simple yet powerful plugin architecture. It requires some experience in C++, using dynamic library (.dll, .so) as well as understanding of fundamental oop concepts, such as interfaces and factories." I think you might want to start with something a little more basic.
Markie
Markie
Hey, thanks for the quick reply!

Quote:
Original post by Sneftel
The first sentence in that article is "This article will guide you through the design of a simple yet powerful plugin architecture. It requires some experience in C++, using dynamic library (.dll, .so) as well as understanding of fundamental oop concepts, such as interfaces and factories." I think you might want to start with something a little more basic.


I thought I had *some* experience in C++ (even if it's not much), using dynamic library (.dll, NOT .so) as well as an understanding of fundamental oop concepts, such as interfaces and factories.
Perhaps not enough though.
Any suggestions to something more basic? Petzold is good, but much too short on DLLs. Noel Llopis and the resources mentioned gloss over or leave out some of the important details.

Mark
Sneftel
Sneftel
Don't overcomplicate it. Have each DLL declare a single exported C function with the signature Plugin* Create(). This returns a heap-allocated object which inherits Plugin. All that stuff is the easy part of plugins, which you're better off spending as little time as possible on, until and unless you specifically need something exotic.
Markie
Markie
Quote:
Original post by Sneftel
Don't overcomplicate it.
Yes, that might have been part of the problem... ;-) Too much reading, too little coding. Gets your head spinning! ;-)

Quote:
Original post by Sneftel
Have each DLL declare a single exported C function with the signature Plugin* Create(). This returns a heap-allocated object which inherits Plugin. All that stuff is the easy part of plugins, which you're better off spending as little time as possible on, until and unless you specifically need something exotic.

The way I understand it now, "new" or "malloc()" create new objects on the heap of an application, regardless if they were called in a main application or from within a plugin and we always pass only pointers to them back and forth across the DLL boundaries. However, for reasons still cryptic to me (C runtime objects?), you must NOT delete or free an object created in a plugin in the main application or vice-versa because for plugins, according to this article: C++ dlopen mini HOWTO
Quote:
You must provide both a creation and a destruction function; you must not destroy the instances using delete from inside the executable, but always pass it back to the module. This is due to the fact that in C++ the operators new and delete may be overloaded; this would cause a non-matching new and delete to be called, which could cause anything from nothing to memory leaks and segmentation faults. The same is true if different standard libraries are used to link the module and the executable.


So really, a plugin with an object needs to supply not only an exported C-style creation-, but also an exported C-style destruction function for that object, correct?

And if you want to create your "product" objects not directly, but with a factory (which makes very good sense for large projects / numbers of objects), you must, in connection with using plugins for "product" object implementation, use "Distributed" or "ABC" factories, right? After all, the whole point of using plugins is to be able to add new objects derived of a base class later on (at run time). And this can only be done dynamically, without knowledge of the new derived object type at compile time with distributed / ABC factories where each derived factory object creates one of the wanted "product" objects. Only with distributed factories can we add code to create new "product" objects as part of the (distributed) factory and in the plugin.

So a proper (cross-platform) plugin which creates "product" objects with a factory needs:
1.) An exported C-style function to create the factory object derived of an Abstract factory Base Class.
2.) An exported C-style function to delete the factory object derived of an Abstract factory Base Class.
3.) The implementation of the factory subclass that is derived of the Abstract factory Base Class and that creates the "product" object.
4.) The implementation of the "product" class derived of the Abstract "product" Base Class.

Is this correct?
Note: Without the UNIX (and cross-compiler?) requirement of exporting only C-style functions, but not C++ classes directly (as quoted below) we could leave away number 1.) and 2.) and export the implementation of the factory class derived of the factory ABC directly.
Again in the Article: C++ dlopen mini HOWTO
Quote:
Another problem with the dlopen API is the fact that it only supports loading functions. But in C++ a library often exposes a class which you would like to use in your program. Obviously, to use that class you need to create an instance of it, but that cannot be easily done.

Solution for plugins:
C++ has a special keyword to declare a function with C bindings: extern "C". A function declared as extern "C" uses the function name as symbol name, just as a C function. For that reason, only non-member functions can be declared as extern "C", and they cannot be overloaded.


Or am I really seeing things too complicated?

Mark
Sneftel
Sneftel
Quote:
Original post by Markie
The way I understand it now, "new" or "malloc()" create new objects on the heap of an application, regardless if they were called in a main application or from within a plugin and we always pass only pointers to them back and forth across the DLL boundaries. However, for reasons still cryptic to me (C runtime objects?), you must NOT delete or free an object created in a plugin in the main application or vice-versa because for plugins, according to this article: C++ dlopen mini HOWTO
Quote:
You must provide both a creation and a destruction function; you must not destroy the instances using delete from inside the executable, but always pass it back to the module. This is due to the fact that in C++ the operators new and delete may be overloaded; this would cause a non-matching new and delete to be called, which could cause anything from nothing to memory leaks and segmentation faults. The same is true if different standard libraries are used to link the module and the executable.
If you're overloading operator new, you know exactly what's going to go wrong there and how to avoid it, but I doubt you're overloading operator new. The real problem, as the article states, is differing versions of the C++ runtime library.
Quote:
So really, a plugin with an object needs to supply not only an exported C-style creation-, but also an exported C-style destruction function for that object, correct?
No, you just need to not call delete/free on the object. But the object itself can have a Destroy member function to do it. Or it can destroy itself when necessary, using reference counting.
Quote:
And if you want to create your "product" objects not directly, but with a factory (which makes very good sense for large projects / numbers of objects), you must, in connection with using plugins for "product" object implementation, use "Distributed" or "ABC" factories, right?


Here's a factory class.
extern "C"{    typedef Plugin* (*PluginCreationFunction)();}class PluginFactory{public:    Plugin* CreatePlugin(std::string const& type)    {        HMODULE hm = LoadLibrary((MODULE_DIR + type + ".dll").c_str());        if(!hm) return NULL;        PluginCreationFunction pcf = (PluginCreationFunction)GetProcAddress(hm, "Create");        if(!pcf) return NULL;        return pcf();    }};

In your MyPlugin.cpp, put extern "c" __declspec(__dllexport) Plugin* Create() { return new MyPlugin; }. That's about it. Actually, that CreatePlugin function up there doesn't really need a PluginFactory to be in. But you seem to want lots of classes, and I suppose you could make MODULE_DIR a member of PluginFactory.
Quote:
distributed factories
Huh? I'm not familiar with that term.

Quote:
C++ has a special keyword to declare a function with C bindings: extern "C". A function declared as extern "C" uses the function name as symbol name, just as a C function. For that reason, only non-member functions can be declared as extern "C", and they cannot be overloaded.
Function mangling is actually a thing you need to worry about, yes. The purpose of extern "C" up there is to keep C++ from mangling function names.
Markie
Markie
Quote:
Original post by Sneftel
Quote:
So really, a plugin with an object needs to supply not only an exported C-style creation-, but also an exported C-style destruction function for that object, correct?
No, you just need to not call delete/free on the object. But the object itself can have a Destroy member function to do it. Or it can destroy itself when necessary, using reference counting.

Ahhha... O.K...
Let me get this right:
This means I can safely put off with #2.):
Quote:
2.) An exported C-style function to delete the factory object derived of an Abstract factory Base Class.
?
We don't need that? We just need one C-style function to create the object? Once it's created we can call a self-made member function of it, or more specifically, we can call its destructor explicitly like so:
--------------------------
pointerToObject->~Object()
--------------------------
Is this right? If so, do we have to use something like:
--------------------------
delete *this;
--------------------------
inside the destructor (or member function) of said object?
Sorry, calling an object's member function / destructor explicitly so it deletes itself is not the trivial case explained in most C++ books.

If that's how things can be done, C++ dlopen mini HOWTO got it wrong when stating that:
Quote:
Next, while still in the module, we define two additional helper functions, known as class factory functions. One of these functions creates an instance of the class and returns a pointer to it. The other function takes a pointer to a class created by the factory and destroys it. These two functions are qualified as extern "C".
and
Quote:
You must provide both a creation and a destruction function;

If this is right, if we DO NOT NEED a destruction function, someone perhaps should tell the guys who wrote this article to correct their text: C++ dlopen mini HOWTO

Thanks for the factory class code snippet. :-)

If I'm not mistaken, a "distributed factory" (Noel Llopis) is what Erich Gamma calls an "Abstract Factory", it's just a factory split up in an "interface" (an abstract base class with (pure) virtual functions) and derived child "Factory" subclasses. One for each type of "product" object which they should create.
This allows adding new product objects by adding new factory subclasses which produce them. The factory subclasses can then be selected polymorphically with factory base class pointers, at run time, which allows introduction of new factory subclasses and thus product objects which they can produce, such as with plugins.

Mark
Sneftel
Sneftel
Quote:
Original post by Markie
We don't need that? We just need one C-style function to create the object? Once it's created we can call a self-made member function of it, or more specifically, we can call its destructor explicitly like so:
--------------------------
pointerToObject->~Object()
--------------------------
Is this right?

Yikes, no. That just calls the body of the destructor function. It doesn't destroy the object.
Quote:
If so, do we have to use something like:
--------------------------
delete *this;
--------------------------
inside the destructor (or member function) of said object?

Also a bad idea. That leads to an infinite loop: delete calls the destructor, which calls delete.
Quote:
Sorry, calling an object's member function / destructor explicitly so it deletes itself is not the trivial case explained in most C++ books.

void MyPlugin::Destroy(){  delete this;}

Quote:
If that's how things can be done, C++ dlopen mini HOWTO got it wrong when stating that:
Quote:
Next, while still in the module, we define two additional helper functions, known as class factory functions. One of these functions creates an instance of the class and returns a pointer to it. The other function takes a pointer to a class created by the factory and destroys it. These two functions are qualified as extern "C".
and
Quote:
You must provide both a creation and a destruction function;

If this is right, if we DO NOT NEED a destruction function, someone perhaps should tell the guys who wrote this article to correct their text: C++ dlopen mini HOWTO
Not really. The only difference there is that their Destroy() function is a free function rather than a member function.

BTW: Read this up to the stuff about multidimensional arrays.
Markie
Markie
Quote:
Original post by Sneftel
Yikes, no. That just calls the body of the destructor function. It doesn't destroy the object.
Also a bad idea. That leads to an infinite loop: delete calls the destructor, which calls delete.

Oops... I guess that just outed me as a big *noob*... ;-)
I had totally forgotten or not fully realized (?) that delete calls the destructor and new the constructor of a class.
*Phew!* Thanks!! :)
That clears up a lot! :))

Quote:
void MyPlugin::Destroy(){  delete this;}


Ahhh... :-)
The best things are so simple sometimes!
Thanks! :)

Quote:
Quote:
If this is right, if we DO NOT NEED a destruction function, someone perhaps should tell the guys who wrote this article to correct their text: C++ dlopen mini HOWTO
Not really. The only difference there is that their Destroy() function is a free function rather than a member function.

O.K., yes, but that doesn't seem so trivial to me. After all, the whole point of them arguing the necessity for an entire exported C-style destroy-function is that... one cannot export classes.
I find it surprising that if a class which is called by a plugin can destroy itself they still argue that one MUST include an exported destroy() function. It seems to me they should have at least considered this option and their text should read something like:
You MUST NEVER destroy a plugin-object from outside a plugin, but you can EITHER export a C-style destroy() function OR you can include a member function with the object which destroys itself with "delete this".
To me, that's how I think things would be stated correctly.
Anyway, thanks a ton for your wisdom and help!!
:-)))

Quote:
BTW: Read this up to the stuff about multidimensional arrays.

Cool! Looks like a lot of answers to good questions! :-)
Thanks! I will read that! In fact, I'm gonna copy or even print the entire page!
:-)

BTW: I think I figured out the problem I had about (abstract) factories and plugin-systems: I think it came form Noel Llopis' book "C++ for Game Programmers". It's a good read, and very interesting but he talks about plugins and a PluginManager in one chapter where he correctly shows a C-style function exported from a DLL "CreatePlugin()" to create an object in the DLL.
Then, in a later chapter he talks about "Object Creation". That's where he talks about the problem of not being able to create objects dynamically and choosing their type at runtime. There he introduces first a factory function with a switch statement switching between possible object types to create, then about an abstract ("distributed") factory to create the objects dynamically. There, every object type gets its own implementation of an abstract creator (interface) class to produce it. An "Entity Factory" then registers object IDs and corresponding creator implementations in an STL "map" container. It can take an objectID, call the corresponding creator implementation and return a corresponding object pointer. - Dynamically, without knowing the objectID at compile time. I figured the two, plugins and abstract factory need to be combined by putting the creator implementations into the DLL together with the object implementation. Then, when I heard that one can only export C-style functions, but not objects, I figured oh god, that means for this kind of a dynamic set-up we need to put the object, the creator implementation AND an exported C-style creator function in every plugin... *phew*
Not so! :-)

Thanks to your help I now realized that we / I don't necessarily need a Creator implementation object as well. Nor an abstract factory. The Plugin-Manager which registers available classes in plugins can act as a normal object creator by using the C-style creator functions exported by the plugins. We don't need an abstract or "distributed" factory, as the Plugin-Manager can act as a normal factory by directly calling exported creator functions.

I realize this still sounds a little "misty" but I think the dust will settle and things will be more clear then, at least to me... ;-)
Thanks again for your help!! :-)

Mark

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.