🔒 Locked

Request for Comments: DLL based architecture for a large game...

Started by jollyjeffers May 12, 2006 at 12:50 PM 33 replies 6.5k views

Original Post

May 12, 2006 at 12:50 PM

[caution] Bit of a lengthy post, but I'd really appreciate your time/thoughts! Evening all, Simply put I've got a large monolithic C++ project/solution that I really need to break up into components. Currently its around 30,000 lines spread over about a 100 files - with the next phase of development likely to triple or quadruple that. I'll fully admit to lazy/bad design, but its starting to get a bit too big, and with more developers due to start work on it soon I can only forsee more problems [headshake] Thankfully the game has a pretty simple architecture - mostly modular/procedural with a few classes thrown in where appropriate. The worst part is that the "engines" are represented as singletons which I've heard aren't too compatable with DLL's - but I'm pretty sure I can get rid of this without much hassle. So, I know theres an awful lot of very experienced programmers kicking around - this is a simple call for comments/experiences. I'm pretty sure I've evaluated all possibilities, but I don't really want to get most of the way through a conversion to find I've missed something obvious and wasted my time. Okay, basic outline is to group each module into its own solution/project (I'm using VC++ 2005 if it matters) and setting it to save/output shared files to a common location. Example:


\Game\
	\Components\
		\Graphics\
			Graphics.sln
			Graphics.vcproj
			Graphics.cpp
		\LogFile\
			LogFile.sln
			LogFile.vcproj
			LogFile.cpp
		\GUI\
			GUI.sln
			GUI.vcproj
			GUI.cpp
		\Config\
			Config.sln
			Config.vcproj
			Config.cpp
		\Utilities\
			Utils.sln
			Utils.vcproj
			Utils.cpp
	\Include\
		Graphics.h
		LogFile.h
		GUI.h
		Config.h
		Utils.h
	\Lib\
		Graphics.lib
		LogFile.lib
		GUI.lib
		Config.lib
		Utils.lib
	\Bin\
		Graphics.dll
		LogFile.dll
		GUI.dll
		Config.dll
		Utils.dll
		Game.exe
	Game.sln
	Game.vcproj
	Game.cpp

The real thing will have many more files, but thats the rough idea. For the core 'Game' solution, each part that relies on one of the componentized modules could have the following directives:

    #include "Include\\Graphics.h"
    #pragma comment( lib, "Lib\\Graphics.lib" )

If the main game as well as all the DLL's are output to the \Bin\ sub-folder then they should all happily find each other on start-up. I've not thought about it extensively, but I'd imagine that all other files (data, config, art, audio...) would be located as a sub-folder of this. Arguably not too clean, but it should work with minimal changes to the current code-base and thats more important to me right now [wink] I've run a few tests with throwing data between a simple console .exe and a .dll to make sure things work and so far I've had no problems with:

Simple function calling
Simple variable sharing
Class instantiation
Simple function pointer / callback usage
Complex functor / member-function callback usage
Passing STL containers around - I use std::wstring quite a bit.

In the case of "Class instantiation" I've done something like this: The definition that is exported externally - all pure interfaces, no concrete classes.

class IBaseControl { ... };
class IButton : public IBaseControl { ... };
bool CreateButton( IButton *p );

And the actual implementation (not visible outside of the DLL) is:

class CBaseControl : public IBaseControl { ... };
class CButton : public CBaseControl, public IButton { ... };

bool CreateButton( IButton *p )
{
    if( NULL != p ) delete p;

    p = new CButton( ... );
}

So far everything has worked just fine, no compile errors/warnings and no runtime problems - desired results all round [grin] But its only a simple proof of concept. I'm not sure if it's so simple as to be hiding any complexities that'll only be revealed when I try and refactor the main codebase. One BIG concern I have (which I'll try testing in a minute) is where components start "talking" to each other. Example scenario: The main game creates and initializes the graphics engine and builds up a GUI by instantiating the various controls imported from the DLL. The application then wants the GUI to be displayed on screen, so calls things like IBaseControl::Draw(). The GUI imports the functionality from the graphics engine (by including the public header and linking to the lib) and makes various calls to render itself appropriately. So we have the main .exe talking to a DLL as well as a DLL talking to a DLL. How many copies of the graphics engine exist? Do I end up with two - one for the main EXE to use and one for the GUI DLL to use? I obviously only want *one* graphics engine, and I want the GUI library to use it when rendering, but to leave ownership/management to the core Game EXE. [smile]

Any comments on what I've proposed - good or bad?
Have I missed anything? Any situations that this will explode?
Could I do better? (Please note that its too late to completely restart my codebase!)

Thanks for reading this far[attention] Jack

jollyjeffers

May 12, 2006 at 12:59 PM

Quote:
Original post by jollyjeffers
In the case of "Class instantiation" I've done something like this:

The definition that is exported externally - all pure interfaces, no concrete classes.
class IBaseControl { ... };class IButton : public IBaseControl { ... };bool CreateButton( IButton *p );
And the actual implementation (not visible outside of the DLL) is:
class CBaseControl : public IBaseControl { ... };class CButton : public CBaseControl, public IButton { ... };bool CreateButton( IButton *p ){    if( NULL != p ) delete p;    p = new CButton( ... );}

Just read through this recent thread and seems that I'd have to make sure that they all link to the correct CRT (/MD and /MDd it seems). I'm likely to force VC++ 2005 on the team because its easy and straight-forward (those who haven't got it can use Express Edition [smile]).

Although, would it make sense to be having a DestroyButton( IButton *p ) to complement the aforementioned CreateButton()?

Therefore potential usage by the game code would replace any new/delete with the appropriate factory methods...

Cheers,
Jack

remigius

May 12, 2006 at 2:48 PM

Well, I'm afraid I won't be of much help, since C++ really isn't my cup of tea. In .NET you'd typically create components in various namespaces to add some semblance of structure to your application. Creating different assemblies (binaries - libraries & exe's alike) is typically only done if you want to make a more generic library.

I don't know how well this translates to C++, but iirc you could do a lot with namespaces too here. The advantage of .NET however is that the namespaces are a bit more powerful, essentially providing all the information you need for 'linking'. This allows you to quite easily move components from you main project later on into subprojects (and hence into different binaries) without any recoding. If this is also the case in C++, I honestly wouldn't know.

Going from your proposed partitioning of code, I'm wondering if you're not going to far with cutting everything up into components. GUI seems to be a subcomponent of Graphics for example, so it would make more sense (to me at least) to put it in the Graphics solution. Also Logfile and Config might just as well be put into the utilities project.

I have a feeling you actually want to have some more technical feedback on C++ solutions for this, but maybe my ramblings may help a bit [smile]

jollyjeffers

May 12, 2006 at 3:15 PM

Thanks for the reply - I'd rate you up for the effort, but you already got my full ++ [wink]

Quote:
Original post by remigius
Creating different assemblies (binaries - libraries & exe's alike) is typically only done if you want to make a more generic library.

I suppose the same could be said in C/C++ as you can therefore create neat, encapsulated and re-usable components. However in this instance its about controlling the build and development process. The end result is relatively unimportant by comparison.

Quote:
Original post by remigius
Going from your proposed partitioning of code, I'm wondering if you're not going to far with cutting everything up into components.

What I posted is just a rough outline for now - I'll refine/verify it in more detail before I actually make any changes.

As for the GUI being part of the graphics - I disagree. But to say that requires more context. Unlike many games, the GUI is a major part of this game (management sim) thus the graphical representation of the GUI is only part of the full equation. The interactions between the GUI, graphics and game pretty much ties the whole game together.

Quote:
Original post by remigius
Also Logfile and Config might just as well be put into the utilities project.

That one makes more sense and would serve to reduce the number of distinct/seperate components. I like it [smile]

Quote:
Original post by remigius
I have a feeling you actually want to have some more technical feedback on C++ solutions for this, but maybe my ramblings may help a bit [smile]

I do appreciate any feedback, buy yes - some more C++/Win32/Windows specific feedback would be better. The .NET framework is a little more helpful/lenient when it comes to sharing/intercommunicating whereas "native" OS-level stuff can be a bit unforgiving - get it wrong and it'll pass the pain directly on to me [smile]

Any further comments?
Cheers,
Jack

JohnBolton

May 12, 2006 at 4:08 PM

Do you have a specific reason for using DLLs? Static libraries would be simpler.

swiftcoder

May 12, 2006 at 4:20 PM

My first suggestion would be to split the 'src' (and possibly the 'include') directories into one subfolder per component. I just did this to one of my own projects when it grew beyond 100 '.cpp' files, as it becomes unmanageable when the IDE forgets some of your files (or similar).

jollyjeffers

May 12, 2006 at 4:56 PM

(EDIT: appreciate the replies, ratings++ for your time/effort [smile])

Quote:
Original post by swiftcoder
My first suggestion would be to split the 'src' (and possibly the 'include') directories into one subfolder per component. I just did this to one of my own projects when it grew beyond 100 '.cpp' files, as it becomes unmanageable when the IDE forgets some of your files (or similar).

Yes, this indeed does make sense. Just trying to reconfigure all of the solution/project files to store their files elsewhere was proving to be "interesting" [smile]

Quote:
Do you have a specific reason for using DLLs? Static libraries would be simpler.

No specific reason initially. I explored the use of static libraries to start with, but I found that they were still inherantly tightly related.

Example:

I'm working on the utilities module; I make some bug fixes, correct some functionality and maybe finish off some pending "todo" items. I rebuild the appropriate DLL and deploy it back to the shared "\bin\" folder. I then run whatever build of the game (or someone else runs their build) and it automagically picks up the changes I've made - provided that the public interface hasn't changed.

Comparatively, using static libraries would require me to at least re-link the updated utilities module with the game before the changes I made become visible. Same with anyone else currently working on the project - they'll need not only the update .LIB file but also to recompile whatever they're testing before they see any of my changes.

I suppose its a fairly minor difference, but to me it boils down to the fact that other developers/testers dont need to know (or specifically do anything) if a components binary changes - its just automagically picked up.

Am I wrong with this, or can you see any reason why static libraries would still be better?

Cheers,
Jack

tstrimp

May 12, 2006 at 5:03 PM

Quote:
Original post by jollyjeffers
I suppose its a fairly minor difference, but to me it boils down to the fact that other developers/testers dont need to know (or specifically do anything) if a components binary changes - its just automagically picked up.

Am I wrong with this, or can you see any reason why static libraries would still be better?

Cheers,
Jack

I assume you're using static libraries to bind the dll to the application? If that 's the case then you should only have to rebuild the binding and application layer if the dll interface changes. Otherwise you should be able to just drop in the new DLL. I'm looking into a similar system myself mainly for automatic updates. I'd much rather only download and replace the dll's that have changed then a huge staticly linked exe.

I'd say dll > static in in this case.

jollyjeffers

May 12, 2006 at 5:09 PM

Quote:
Original post by tstrimp
I assume you're using static libraries to bind the dll to the application? If that 's the case then you should only have to rebuild the binding and application layer if the dll interface changes.

Yes, this is exactly the situation.

The main executable is compiled against the public header and lib file, both of which are pretty much fixed/static at this stage in development. However, the implementation is changing fairly regularly - mostly bug fixes, but also the occasional improvement/change.

Quote:
Original post by tstrimp
I'm looking into a similar system myself mainly for automatic updates.

A nice advantage as I see it, but I've little use for updates beyond bug patching.

Quote:
Original post by tstrimp
I'd say dll > static in in this case.

Okay, but I'm still very interested in hearing of anyones experiences in using/implementing this sort of setup.

On paper it looks great, but in practice I'm just wondering whether it'll solve some problems but introduce others [oh]

Cheers,
Jack

Magius

May 12, 2006 at 5:23 PM

Quote:

Original post by jollyjeffers
In the case of "Class instantiation" I've done something like this:

The definition that is exported externally - all pure interfaces, no concrete classes.

class IBaseControl { ... };
class IButton : public IBaseControl { ... };
bool CreateButton( IButton *p );

And the actual implementation (not visible outside of the DLL) is:

class CBaseControl : public IBaseControl { ... };
class CButton : public CBaseControl, public IButton { ... };

bool CreateButton( IButton *p )
{
if( NULL != p ) delete p;

p = new CButton( ... );
}

The first thing that jumps out at me here is COM, which was a very elegant solution for its time and in practice works quite well, even in the case of using COM-like concepts in the absence of the COM framework. To solve the component problem, COM components are only supposed to communicate via interfaces. The objects implementing the interface are created by the COM server and exposed back to the client via the interface. The objects themselves are generally created by a factory (your Create, in this case) and destroyed through a reference counting mechanism (i.e. AddRef / Release). The reference counting solution has it's problems, but it's easy to implement and you can at least avoid circular references in your own code (the public API is usually a different story)...

COM was and still is a very successful technology and the concepts still used widely today. Even better, the C factory functions can be exported with no worries of name mangling and other problems that can rear their ugly head when using dlls. I find this method of implementation simple and elegant so long as your public API is, for the most part, very stable.

Magius

jollyjeffers

May 12, 2006 at 5:37 PM

Being a DirectX programmer I'm fairly familiar with using COM, although I've not really spent much time actually implementing stuff using COM (or COM concepts).

I think COM might be overkill for this purpose though, but the factory methods might well be the way forward.

I'm putting together a demo framework right now. If people are interested I could post the files? If not to answer my questions, but as much for anyone else interested in the same problem [smile]

Cheers,
Jack

FBMachine

May 12, 2006 at 5:42 PM

I use a similar approach in my projects that I picked up from working on Unreal Engine based games. Basically my projects are setup with a directory structure like:

Project ( contains [project].sln, solution contains all subprojects )  - WinExe ( app entry project, contains WinExecutable.vcproj, creates exe )          - Src ( source obviously )          - Inc          - Debug/Release ( contains intermediate files )  - Renderer/Whatever ( contains [whatever].vcproj, creates dll )          - Src          - Inc          - Debug/Release  - System/Bin ( target directory for the exe and dll's )

And the project layout in Visual Studio basically mirrors this structure. It keeps things clean on huge projects, and using dll's saves you having to re-link as you said.
Daniel

Magius

May 12, 2006 at 5:42 PM

I can say from experience that these concepts work very well in practice (aside from the whole circular reference thing but again, it's more a problem for people that don't actually know how to use the public API than your implementation). I find component based architectures very elegant to use. Of course, the full COM framework is heavier than what you need, but there isn't any reason why the basic concepts of the framework wouldn't work well for you.

Magius

CTar

May 12, 2006 at 5:52 PM

Quote:
Original post by jollyjeffers
The main game creates and initializes the graphics engine and builds up a GUI by instantiating the various controls imported from the DLL. The application then wants the GUI to be displayed on screen, so calls things like IBaseControl::Draw(). The GUI imports the functionality from the graphics engine (by including the public header and linking to the lib) and makes various calls to render itself appropriately.

So we have the main .exe talking to a DLL as well as a DLL talking to a DLL. How many copies of the graphics engine exist? Do I end up with two - one for the main EXE to use and one for the GUI DLL to use?

I obviously only want *one* graphics engine, and I want the GUI library to use it when rendering, but to leave ownership/management to the core Game EXE. [smile]

Lets say you have the three project 'Game', 'GUI' and 'Renderer'. 'Renderer' wouldn't depend in any way on 'Game' or 'GUI'. 'Game' would need both the interfaces and implementations of 'Renderer' and 'GUI'. 'GUI' wouldn't own the renderer so it won't need the actual implementation only the interface. So with the GUI you can get away with just '#include'ing the header file for the renderer, and not linking to the .lib.

The GUI should store an IRenderer pointer somewhere which it uses for anything graphics related, of course the IRenderer pointer should be provided by 'Game' which owns the renderer. The IRenderer pointer could be stored in many places, the most OO-correct way would most likely be to have every class derived from IBaseControl to store a pointer to a gui system object, this gui system would then store a pointer to the renderer used; or maybe the IBaseControl could just have a pointer to the IRenderer (with no GUI system). You mentioned singletons, so if you're sure there will only be one GUI system you could let IBaseControl have a static pointer to the IRenderer. If the renderer is also a singleton, then you could have a potential problem. I'm not too sure, but I think there would be created two seperate singletons if you link to the renderer from both 'GUI' and 'Game', this would ofcourse result in wrong behavior. The best solution I could think of (besides removing the singletons) would be for 'Game' to get an actual pointer to the IRenderer (via GetInstance or what you call that function), and pass it on to the GUI, just like you would have done if it wasn't a singleton. This could introduce new problems with threading though, since the singleton would only check if the renderer is currently accessed in the GetInstance function, but the GUI might also access it.

I would suggest you to get rid of the singletons, I don't know if it's too late in the project, but if possible you should get rid of them.

jollyjeffers

May 12, 2006 at 6:13 PM

Thanks for the continued comments, Ratings++ where possible!

Quote:
I use a similar approach in my projects that I picked up from working on Unreal Engine based games. Basically my projects are setup with a directory structure like:

So have you actually got seperate projects/solutions, or is it all grouped into one "master" solution?

I experimented with the "master" solution idea in VStudio - it was very easy to set up, but seemed too tightly connected. Ideally I want to break them apart completely, VStudio still wanted to load/check/compile all of the sub-projects regardless of whether I was working on them. I'm the software architect and multimedia programmer - I DONT want the other developers to know about things that dont concern them. Not in an unfriendly way of course, but just a case of "use the public interfaces, if it goes wrong its not your problem - talk to me!" Or possibly "what you dont know wont hurt you" [grin]

Quote:
Of course, the full COM framework is heavier than what you need, but there isn't any reason why the basic concepts of the framework wouldn't work well for you.

Agreed, I shall give this some more thought. Thanks!

Quote:
I would suggest you to get rid of the singletons, I don't know if it's too late in the project, but if possible you should get rid of them.

Its not too late. The actual usage of the singletons is largely hidden from the codebase (fugly use of #define [wink]) so I think I can swap it out without too much hassle.

You suggest passing an IRenderer from 'Game' to 'GUI' components - I like the sound of that, but I was thinking about implementing the components in a procedural way - not exporting classes from the DLL's. I got thinking and realised that there isn't really any need for classes. Private data is already hidden via the DLL/interface mechanism, there is no inheritance, there is no multiple instances... why a class?

As previously mentioned, I'm trying to throw together a test case right now with multiple components and trace messages. See whether it fits together in a half-way sane method [grin]

I really appreciate the comments - thanks!
Jack

Magius

May 12, 2006 at 6:24 PM

We write very large applications where I work, most using Visual Studio. The subsystems are separated into separate source control entities (mostly because of a large amount of global collaboration) and separate solutions for each subsystem. Even subsystems themselves tend to have multiple solutions if the structure warrants it, but this depends on a number of factors, including code isolation, configuration management process, and taste of the subsystem lead. There is really no "best way" to do this, so it might make sense to invest a small amount of time into some sort of process set-up if you plan to work in a reasonably sized team (too much process is overkill). The system actual builds using a separate build mechanism (you can use MSBuild if you want to set up automated build tasks) that builds both nightly and at specified configuration management intervals. In your case, I might suggest you base solutions on team size, component / subsystem parts, and estimated build times of each component. Experimentation and tuning tends to work fairly well in smaller projects (i.e. separate out when the situation warrants it).

Magius

FBMachine

May 12, 2006 at 6:50 PM

Quote:
Original post by jollyjeffers
So have you actually got seperate projects/solutions, or is it all grouped into one "master" solution?

I experimented with the "master" solution idea in VStudio - it was very easy to set up, but seemed too tightly connected. Ideally I want to break them apart completely, VStudio still wanted to load/check/compile all of the sub-projects regardless of whether I was working on them. I'm the software architect and multimedia programmer - I DONT want the other developers to know about things that dont concern them. Not in an unfriendly way of course, but just a case of "use the public interfaces, if it goes wrong its not your problem - talk to me!" Or possibly "what you dont know wont hurt you" [grin]

Yes, seperate projects, but one master solution. Much easier for project-wide debugging, and sometimes it's useful to have the source to other peoples components for various reasons. Seems kind of silly to hide your code from your co-workers. You can just do exclusive check-outs if you don't want people messing with your stuff. :)
Also, there's nothing stopping you from building only the current subproject you're working on, you don't have to recompile everything every time.
And another downside I can see if all your subcomponents are in seperate solutions, if you want to switch configurations ( debug/profile/release/whatever ) project-wide, you'd have to have all the projects open and build them seperately.
Daniel

Magius

May 12, 2006 at 6:57 PM

Separate projects definitely. FBMachine makes a good point here:

Quote:

You can just do exclusive check-outs if you don't want people messing with your stuff. :)

This is often overlooked in small teams - if you manage the source control correctly, exclusive checkouts can provide a great means of enforcing access. In general, more than one person working on the same exact code is an indication (to me at least) that responsibilities weren't separated out properly at the beginning or that the design was not thought out enough up front (perhaps the public interface wasn't fine-grained enough).

Quote:

And another downside I can see if all your subcomponents are in seperate solutions, if you want to switch configurations ( debug/profile/release/whatever ) project-wide, you'd have to have all the projects open and build them seperately.

If the project is big enough, chances are there is a process in place for this. Separate build mechanisms that run independent of Visual Studio being open or not (i.e. command line builds) can easily choose the configuration at a master level and distribute this configuration down to the solution level (and project) when building the files. It takes some work up front, but even a simple command line .bat file can do this quite easily. If you don't mind me asking, what are your current plans in terms of team size?

Magius

jollyjeffers

May 12, 2006 at 7:25 PM

Quote:
Original post by FBMachine
Yes, seperate projects, but one master solution. Much easier for project-wide debugging, and sometimes it's useful to have the source to other peoples components for various reasons.

Yup, understood. I have a fairly heavy-weight logging, statistics and general monitoring setup, so I'm not hugely fussed about project-wide development.

We have a very distributed team - different timezones, schedules etc...etc... thus independence and "co-existance" is going to be useful. For example, if I can get on with fixing up and improving the engine components without paying any attention to the game code then thats great [smile]

Quote:
Original post by FBMachine
Seems kind of silly to hide your code from your co-workers.

Yup, but the code wont be hidden as such, more that its a nice boundary where "the other side of the line does not concern you". Not that they can't cross it, nor that they cant check it out, more that if it breaks they should come shouting at the respective owner.

Quote:
Original post by FBMachine
And another downside I can see if all your subcomponents are in seperate solutions, if you want to switch configurations ( debug/profile/release/whatever ) project-wide, you'd have to have all the projects open and build them seperately.

Yes, this is a problem I had considered.

Changing the individual components to build a ".dll" and a "_d.dll" (the latter _d for the debug build) and various #ifdef _DEBUG statements should help out. Not perfect, but at this stage its not my primary concern [wink]

Quote:
if you manage the source control correctly, exclusive checkouts can provide a great means of enforcing access.

I'm not responsible for the server (+software) management, but I'm checking out the options now.

Quote:
f you don't mind me asking, what are your current plans in terms of team size?

probably 2-3 developers working on the code I've been discussing in this thread. Maybe expanding to more (keep your eye on 'Help Wanted' over the next few months if you're up for some work [wink]). Then another 10-15 people who are doing project-related work (e.g. art, audio, design...) and then another 20 or so doing testing (who'll need builds etc..)

Cheers,
Jack

FBMachine

May 12, 2006 at 7:34 PM

Quote:
Original post by Magius
If the project is big enough, chances are there is a process in place for this. Separate build mechanisms that run independent of Visual Studio being open or not (i.e. command line builds) can easily choose the configuration at a master level and distribute this configuration down to the solution level (and project) when building the files.

True. In fact Incredibuild can do this, which we do use at work. I just use it simply through the Visual Studio integration though since we use a master solution, so I have a narrow view on this. :) But yeah, the master solution with subprojects structure that Unreal uses suits our 8 man programming team ( and my personal projects ), but ymmv for larger teams/projects.
Daniel

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.