How should a DLL be designed?

Started by
8 comments, last by Dawoodoz 4 years, 5 months ago

I know how DLL works, how it is created and how it is imported at IDE level, but I don't know anything about its design.

Imagine that I want to abstract the Engine project into a DLL, and use it from another project called Game. What will be the points through which Game accesses the Engine classes and functionalities?

Should a single access point be created through an Engine object that has access to everything and to which I ask for things, or create several essential objects and access to functionalities is provided through each of them, such as Engine, Actor / GameObject, Level?

Thank you.

Advertisement

For Windows specifically, using DLL's properly can be a notable challenge.  The problems tend to fall in two categories: memory management and content discovery.  Memory management specifically causes the most problems because it is silent but deadly when you mess up.  Basically a DLL in Windows is just another executable which is mapped into the same memory space as the caller and even has it's own main function (DLLMain) as an optional item.  The impact of this is that Windows has several different ways of integration with the runtime memory manager: static versus shared, single threaded versus multithreaded, debug versus release, unicode variations and probably more I'm forgetting.  Mixing these different memory manager types works for allocations but when it comes time to free the memory if you call from the wrong library, bad things happen (tm)...  Simply stating that everything will use a single variation of memory manager at all times is not going to fix things in many ways.  For instance if you use static runtimes you still end up with two separate memory heaps.  Additionally you can't always guarantee that 3rd party libraries you may use can be built in the same manner, so sometimes you are stuck.

So, a first goal of utilizing DLL's in Windows is making sure you can fix this problem or absolutely guarantee that no memory ownership is ever going to cross from one side of the boundary to the other.  There are a number of potential solutions to this, a common one is to write your DLL content to use an installable override for alloc/free, another would be making all return types opaque handles and a third would be using a COM like reference counted solution.  Depending on goals, the way you approach this is up to you though in general the reference counting solution has advantages for the second part of your question.

Discoverability can be either very easy or very hard depending on goals.  If you wish to use classes and C++ over the DLL boundary there are a number of challenges involved.  First, linkage is compiler specific meaning that in Windows you have to tag things up with declspec(dllexport) everywhere, guarantee the memory management is handled properly (generally speaking you must use the DLL variations of the runtime) and you must still be aware of side issues such as throwing exceptions being illegal over the boundary.  (Pretty sure the exception issue is still there, haven't checked in a long time though.)  I really don't suggest trying to use C++ classes over the DLL boundary in general, it is just problematic in too many ways, this also means passing things like std::vector in the interfaces is highly dubious.

The most common solution to discoverability involves writing factory classes for anything you need to be shared over DLL boundaries and then exporting one or two functions from every DLL.  This is the plugin pattern such that your exe loads the DLL, finds the single function and then calls it so it can register it's content into the factory(s).  In order to make this useful there are several potential solutions, but as mentioned I favor the ref count solutions and a COM like pattern:


// Public API stuff...
struct MyStuffInterface
{
  virtual int32_t AddRef() = 0;
  virtual int32_t Release() = 0;

  virtual int32_t Add(int32_t rhs) = 0;
};

MyStuffInterface* (*Creator)(void);
struct Factory
{
  virtual int32_t Install(Creator* creator) = 0;
  virtual MyStuffInterface* Create() = 0;
};

// Private DLL implementation.
class MyStuff : public MyStuffInterface
{
  ... trivial implementation details ...
};

declspec(dllexport) int Install(Factory* factory)
{
  factory->Install([]() { return new MyStuffImpl(); });
  return 1;
}

If you don't like the C++ in this, you can write a compatible variation in C fairly trivially and in fact if you stick to pure C style arguments in the interfaces, you sidestep entire swatches of potential DLL hassles.  The benefit is that you can easily expose entire API's from the DLL without the calling side knowing the internals of the DLL except the single "Install" function.

Having said all this, there are many more ways to go, this is just my favorite approach because it solves a lot of different issues in a clean and concise manner.  Well, as clean as you get with Windows and the horrible way DLL's work in the totally mixed up environment which is Windows....  :P

57 minutes ago, All8Up said:

For Windows specifically, using DLL's properly can be a notable challenge.  The problems tend to fall in two categories: memory management and content discovery.  Memory management specifically causes the most problems because it is silent but deadly when you mess up.  Basically a DLL in Windows is just another executable which is mapped into the same memory space as the caller and even has it's own main function (DLLMain) as an optional item.  The impact of this is that Windows has several different ways of integration with the runtime memory manager: static versus shared, single threaded versus multithreaded, debug versus release, unicode variations and probably more I'm forgetting.  Mixing these different memory manager types works for allocations but when it comes time to free the memory if you call from the wrong library, bad things happen (tm)...  Simply stating that everything will use a single variation of memory manager at all times is not going to fix things in many ways.  For instance if you use static runtimes you still end up with two separate memory heaps.  Additionally you can't always guarantee that 3rd party libraries you may use can be built in the same manner, so sometimes you are stuck.

So, a first goal of utilizing DLL's in Windows is making sure you can fix this problem or absolutely guarantee that no memory ownership is ever going to cross from one side of the boundary to the other.  There are a number of potential solutions to this, a common one is to write your DLL content to use an installable override for alloc/free, another would be making all return types opaque handles and a third would be using a COM like reference counted solution.  Depending on goals, the way you approach this is up to you though in general the reference counting solution has advantages for the second part of your question.

Discoverability can be either very easy or very hard depending on goals.  If you wish to use classes and C++ over the DLL boundary there are a number of challenges involved.  First, linkage is compiler specific meaning that in Windows you have to tag things up with declspec(dllexport) everywhere, guarantee the memory management is handled properly (generally speaking you must use the DLL variations of the runtime) and you must still be aware of side issues such as throwing exceptions being illegal over the boundary.  (Pretty sure the exception issue is still there, haven't checked in a long time though.)  I really don't suggest trying to use C++ classes over the DLL boundary in general, it is just problematic in too many ways, this also means passing things like std::vector in the interfaces is highly dubious.

The most common solution to discoverability involves writing factory classes for anything you need to be shared over DLL boundaries and then exporting one or two functions from every DLL.  This is the plugin pattern such that your exe loads the DLL, finds the single function and then calls it so it can register it's content into the factory(s).  In order to make this useful there are several potential solutions, but as mentioned I favor the ref count solutions and a COM like pattern:



// Public API stuff...
struct MyStuffInterface
{
  virtual int32_t AddRef() = 0;
  virtual int32_t Release() = 0;

  virtual int32_t Add(int32_t rhs) = 0;
};

MyStuffInterface* (*Creator)(void);
struct Factory
{
  virtual int32_t Install(Creator* creator) = 0;
  virtual MyStuffInterface* Create() = 0;
};

// Private DLL implementation.
class MyStuff : public MyStuffInterface
{
  ... trivial implementation details ...
};

declspec(dllexport) int Install(Factory* factory)
{
  factory->Install([]() { return new MyStuffImpl(); });
  return 1;
}

If you don't like the C++ in this, you can write a compatible variation in C fairly trivially and in fact if you stick to pure C style arguments in the interfaces, you sidestep entire swatches of potential DLL hassles.  The benefit is that you can easily expose entire API's from the DLL without the calling side knowing the internals of the DLL except the single "Install" function.

Having said all this, there are many more ways to go, this is just my favorite approach because it solves a lot of different issues in a clean and concise manner.  Well, as clean as you get with Windows and the horrible way DLL's work in the totally mixed up environment which is Windows....  :P

Wow, thank you very much. This is a lot of information, I will try to digest it.

I'm working with DLLs or better static libs since I first redesigned my engine because the modular approach feeled better to me like the monolithic one project solution. This way I can design and compile my modules in a way that allows them to stay apart from each other, especially if you work in a team this is very usefull.

My approach of deciding what belongs where is simple,

  • DLLs/libs provide utility classes so they have a public interface.
  • Each class stays on it's own except for needed dependencies to other classes (like when defining a member of certain type)
  • Put as much code as you can in header files and inline it (so the compiler can optimize calls)
  • Put code that is specific to certain circumstances in code files (like the implementation of an API function for different platforms)

and last but not least, I maintain a hirachy system. If a class or function is used in multiple modules, then it's hirachy level is increased so both modules have a dependency to the new module. This way I can build a dependency pyramide. My custom build-tool also outputs a dependency graph that contains all classes in my projects to help decoupling modules and interfaces from each other

51 minutes ago, Shaarigan said:

I'm working with DLLs or better static libs since I first redesigned my engine because the modular approach feeled better to me like the monolithic one project solution. This way I can design and compile my modules in a way that allows them to stay apart from each other, especially if you work in a team this is very usefull.

My approach of deciding what belongs where is simple,

  • DLLs/libs provide utility classes so they have a public interface.
  • Each class stays on it's own except for needed dependencies to other classes (like when defining a member of certain type)
  • Put as much code as you can in header files and inline it (so the compiler can optimize calls)
  • Put code that is specific to certain circumstances in code files (like the implementation of an API function for different platforms)

and last but not least, I maintain a hirachy system. If a class or function is used in multiple modules, then it's hirachy level is increased so both modules have a dependency to the new module. This way I can build a dependency pyramide. My custom build-tool also outputs a dependency graph that contains all classes in my projects to help decoupling modules and interfaces from each other

Hi, Could you tell me why are build tools written in C #? I see it in your Forge and I have seen it in Unreal(Unreal Build Tool). I see that it is a very complete tool.

C# is a language with a rich feature set and without the need of managing memory by one's own. This is a big plus because those tools often have to be robust and their development time should be very low (if you are motivated enougth). UBT is created in C# because it supports on-demand compilation of the module settings written in C# also. UBT creates an assembly from it, grabs the classes via reflection and integrates them into the build process.

Forge is written in C# for the same reasons, we have also .Build.cs files to configure the project, they are unless Unreal, build into a Mixins (extensions that are connected to well defined points in the tool), but we are also using a quick-setup. The script locates .Net Framework 4 on your PC and calls CSC.exe, the old C# compiler to compile the initial version of Forge before you can start coding.

The upcomming version of Forge will be even more flexible due to a built-in compiler to translate Blueprint-like visual developed pipelines into C# code loaded as Plugin into it

1 hour ago, Shaarigan said:

C# is a language with a rich feature set and without the need of managing memory by one's own. This is a big plus because those tools often have to be robust and their development time should be very low (if you are motivated enougth). UBT is created in C# because it supports on-demand compilation of the module settings written in C# also. UBT creates an assembly from it, grabs the classes via reflection and integrates them into the build process.

Forge is written in C# for the same reasons, we have also .Build.cs files to configure the project, they are unless Unreal, build into a Mixins (extensions that are connected to well defined points in the tool), but we are also using a quick-setup. The script locates .Net Framework 4 on your PC and calls CSC.exe, the old C# compiler to compile the initial version of Forge before you can start coding.

The upcomming version of Forge will be even more flexible due to a built-in compiler to translate Blueprint-like visual developed pipelines into C# code loaded as Plugin into it

This information is very valuable to me. Thanks for explaining.

It could be not exactly that you expect; but I would like to present my tool for creation DLLs by using C++ and exposing pure C interface + again C++ wrapped interface. https://github.com/PetrPPetrov/beautiful-capi

Keep it simple and anyone can call your library
The interface is on a C level, so think procedural first. Not all functions need a reference to a whole object, which can also increase code reusability internally. Any type pointers in a DLL will limit the set of languages that can call the DLL. For maximum portability across languages, only use 32-bit floats and 32-bit signed integers. Languages for beginners don't have 32-bit unsigned integers because it often leads to underflow.

ActiveX is more limited
If you plan to support ActiveX, you might need a global state to return multiple values, so design the API to receive data before use rather than holding and manipulating data. Don't try to go around the safety of slow ActiveX calls for the sake of performance, try to design the API to reduce the number of calls in the first place.

Calling from a safe language? Make the library safe to call
If you target a safe language such as Visual Basic, C#, Java or Python, make sure to use indices to address class objects with reference integrity, so that no invalid calls from the safe language can't cause an unknown crash without any informative message. Prepare a message queue from where the caller can receive information about the crash and choose how to present the message.

Server based object model
When using global functions to access objects, you can achieve a higher encapsulation than plain C++ classes by taking care of memory management for the caller. The overhead of heap allocation can be overcome by making your own memory allocator that improves cache locality instead. This will only lead to a global state if you fail to implement it according to the isolation principles, which is most likely to happen from bugs in garbage collection. Doing something in caller A should not cause undefined side-effects for caller B. If you have a global list of renderable objects, just let them be allocated there without doing anything until you call them. If they belong to a world context, it's okay to have side effects tied to the world object.

This topic is closed to new replies.

Advertisement