I'm considering the best approach at the moment for writing a DLL-based plugin system for a 3D modelling program I am working on. I've asked a few questions here before and think I have come up with a possible strategy but I'm finding it quite hard to get any comprehensive information off the internet so wanted to outline it here and see if there are any problems or better solutions before I get too far down the road.

I'd originally wanted to distribute a static lib that users link into their DLLs, but of course then if the static lib changes, any existing DLLs need to be recomplied, which I don't want if possible.

What I was thinking was to interface between the main application and user DLLs via another DLL, linked to both the exe and to user DLLs. I currently have the main model class which needs exposing to the DLLs represented by a normal class, using STL containers and all that stuff that won't play nice across boundaries, in the main application itself.

I figure if I wrapped that model class into a DLL, and exposed all the functionality via C-syntax free functions that take a pointer to the model and return vertex positions, face indices etc, including free functions to create and destroy the models, surely I could link both the application and any user DLLs to this? Then I would be able to change the internals of this intermediary DLL, or even expand its interface to include new methods, without needing to recomplie any existing user DLLs that existed?

Internally the application could use this interface to create and manipulate models, and the app could then pass the pointer obtained from the intermediate DLL into the user DLLs so they could also call this intermediate interface with the pointer.

Is this feasible? Thanks.

I figure if I wrapped that model class into a DLL, and exposed all the functionality via C-syntax free functions that take a pointer to the model and return vertex positions, face indices etc

Yes, that kind of free functions can have ABI (application binary interface) compatibility, easier than exposing classes.
An alternative way is just to expose interface, which is a collection of pure virtual functions. This tech is wide used in Windows COM.
Also be careful with STL. Different DLLs maybe compiled with different STL which is ABI incompatible.

However, developing DLLs plugins in C++ (I assume you are using C++) forces you to have a lot of considerations, such as ABI compatibility, how to pass object, etc.
Did you ever consider some alternative way? Such as use some embedded script language as the glue and use C++ to build the underlying system?

I really don't see why you need to use an intermediary DLL here. Just define your model class as deriving from an abstract base class and pass pointers of those to the DLL.

Hmm, I get it. The DLL just has a header that defines the interface and the interface is actually implemented within the application, then the DLL code obviously can't instatiate an instance of this abstract class (which is correct) but all of the methods that get called happen within the exe rather than the DLL or a static lib. Interesting, thanks. Think I've been over-complicating this somewhat.

Does this approach cope well with a DLL compiled with a different compiler to the one used to compile the application?

Thanks to wqking for the comment as well. I've thought about scripting languages but would rather have a DLL plugin system because it would then be possible to write various DLL's that provided support for any number of scripting languages. The reverse is obviously not the case.

On Windows, the layout of the vtable for an abstract base class is de facto standardized - it's how COM is defined for C++, so if your compiler supports COM and the other compiler supports COM it should be able to support your application.

Awesome, thanks for the information.

On Windows, the layout of the vtable for an abstract base class is de facto standardized - it's how COM is defined for C++, so if your compiler supports COM and the other compiler supports COM it should be able to support your application.

+1,

@Aardvajk, I suggest you look at some basic COM. The interface and reference count memory manage is good to work across DLLs.