How to organize a C++ project and its external libraries

Started by
6 comments, last by BurrickSlayer 10 years, 8 months ago
Greetings!

I just registered on the forums after finding a thread with a question similar to mine. I decided to create a new thread to not hijack the other thread and because my question is slightly different.

My C++ project files are under version control so I can work from different locations easily. The external libraries I use (currently only SFML and pugixml) are not under version control yet though. I would like to change that, so that the repository contains everything that is needed to build the project. I also would like to keep it more platform and compiler independent than it currently is (it's optimized for Windows + MinGW at the moment).

So, how can I achieve that?

My first idea was to organize my project like that:

project
    - extlibs
        - include
            - SFML // same headers for all platforms and compilers
            - pugixml // headers + source, no bins, no libs
        - win32
            - vs2012
                - SFML
                    - bin
                    - lib
            - gcc-4.7-tdm-sjlj
                - SFML
                    - bin
                    - lib
            - gcc-4.7-mingw-dw2
                - SFML
                    - bin
                    - lib
        - win64
            - ...
    - src

Is this project layout a good approach?

Suggestions very welcome.
Advertisement

Hi and welcome. So, the big suggestion I have is to avoid modifications to the external libraries. This does of course mean some pain in terms that without modification each external library will have different include locations, build output etc but in the long run it makes updating the libraries considerably easier. Additionally, not making modifications to the libraries means less hassle with modifying the build systems and potentially having to modify the source itself to adjust paths and such. I realize from experience that reorganizing the externals is tempting and seems like a good idea so it matches your other code better but it has been my experience that it simply causes too many problems down the road.

While I use CMake and do things in a very modular manner you might want to take a look at the tutorials project I use for writing articles. It's not a perfect setup but it may give you some ideas. You can find it on Google Code here. In terms of the other thread, the layout is about what I would suggest also for various reasons. The primary problem with team work for most projects is generally caused when multiple people are working in shared code, if you have a lot of smaller libraries, it reduces the constant cross purpose modifications which generate merge conflicts. Anyway, hope this gives you some thoughts on how to move forward.

Perhaps consider putting external lib even higher up the chain, as you could have multiple project use the same external SFML library.

something like:

./dev

./dev/external/sfml

./dev/external/box2d

./dev/projects/pong

./dev/projects/tetris

Thank you both for your suggestions!

@nhatkthanh

This is similar to my current setup. All my external libraries are stored in D:\Libs\. In my project root directory I have a folder named paths, that contains symbolic links to the required external libraries.

It works for me, so I'll probably keep it that way if it turns out to be the best option.

@AllEightUp

Your tutorial project is very helpful, thanks. I use CMake, too (along with QT Creator). I'm not very adept in it, yet, so your CMake scripts are also interesting for me.

I'm not entirely sure what you mean with "avoiding modifications to the external libraries", though. Were you referring to how I split up SFML in separate directories for headers and platform- and compiler-specific bins and libs? I was inspired by way SFML handled external libraries, as the SFML repository contains DLLs and libs for OpenAL and libsndfile. They're added like this in /src/SFML/CMakeLists.txt:



# let CMake know about our additional libraries paths (on Windows and OS X)

if (WINDOWS)

    set(CMAKE_INCLUDE_PATH ${CMAKE_INCLUDE_PATH} "${PROJECT_SOURCE_DIR}/extlibs/headers")

    if(COMPILER_GCC)

        if(ARCH_32BITS)

            set(CMAKE_LIBRARY_PATH ${CMAKE_LIBRARY_PATH} "${PROJECT_SOURCE_DIR}/extlibs/libs-mingw/x86")

            set(CMAKE_LIBRARY_PATH ${CMAKE_LIBRARY_PATH} "${PROJECT_SOURCE_DIR}/extlibs/bin/x86")

        elseif(ARCH_64BITS)

            set(CMAKE_LIBRARY_PATH ${CMAKE_LIBRARY_PATH} "${PROJECT_SOURCE_DIR}/extlibs/libs-mingw/x64")

            set(CMAKE_LIBRARY_PATH ${CMAKE_LIBRARY_PATH} "${PROJECT_SOURCE_DIR}/extlibs/bin/x64")

        endif()

    elseif(COMPILER_MSVC)

        ...



I thought I could do it in a similar fashion.

Or should I rather use the script FindSFML.cmake instead? I just recently payed attention to this script.


The external libraries I use (currently only SFML and pugixml) are not under version control yet though. I would like to change that, so that the repository contains everything that is needed to build the project. I also would like to keep it more platform and compiler independent than it currently is
Yeah I prefer to make an "external" directory, inside my project's directory, which contains all the dependencies, rather than having all my dependencies existing independently outside of my project directory.

The reason for this is simply configuration management and quality assurance. If you're relying on independent external libraries, then different people on your team (or different PC's of your own) might produce different results, due to those different people/PCs having different versions of the external libraries installed.

By bundling them into your own project, you ensure that it is always built the exact same way with the exact same code.

The layout I'd normally use would be leave the external libraries alone. My project files (CMake etc) will be a bit messy as each external library will use a different layout for it's own files, but it's easier to just leave their distro alone and deal with this slightly messy config on my end, rather than try to reorganize their files to the way I'd prefer every single time I update to a new version of their library...

project/msvc/...     compiler specific stuff (projects/workspaces/etc)
project/src/...      code that's internal to the library
project/include/...  code for users of the library
project/bin/...      where it's built to
project/external/... third party libraries required
e.g.
project/external/foo/... the foo distribution *untouched*
project/external/foo/include
project/external/foo/src
project/external/foo/lib/x86windows
project/external/bar/... the bar distribution *untouched*
project/external/bar/includes
project/external/bar/sources
project/external/bar/bin/win32

@AllEightUp

Your tutorial project is very helpful, thanks. I use CMake, too (along with QT Creator). I'm not very adept in it, yet, so your CMake scripts are also interesting for me.

I'm not entirely sure what you mean with "avoiding modifications to the external libraries", though. Were you referring to how I split up SFML in separate directories for headers and platform- and compiler-specific bins and libs? I was inspired by way SFML handled external libraries, as the SFML repository contains DLLs and libs for OpenAL and libsndfile. They're added like this in /src/SFML/CMakeLists.txt:

I thought I could do it in a similar fashion.

Or should I rather use the script FindSFML.cmake instead? I just recently payed attention to this script.

In regards to SFML, it makes sense for that library to supply the includes/libs/dll's in a cleaned up manner as the point of SFML is easy use and distribution. This is different than writing a game though where the focus is not on easy distribution so much as it is on getting the job done. Let's say it takes a couple hours to build all the variations of SFML, make copies of the includes, put the binaries where you want them etc, that's a couple hours you could be coding on the game instead of fiddling with trivialities. With the solution I use in the tutorials project, if I upgrade SFML I just drop it into the extern folder and point the listfile to the new location, delete the old version and keep working on my game. 5 minutes (ok, 10 if you count the extra build time for the new version :)) versus a couple hours.

As to making no modifications, I mean that it should be literally a matter of downloading the new source package, unzip to the extern directory and update the CMake list files to point at the new version. Anything beyond that, or perhaps very trivial modifications, is just too much of a hassle generally. The libraries which come with SFML are not exactly my favorite thing, but then again, building FreeType2 is a pain in the ass (on Windows at least) and SFML is coded against those versions so I have no problem using them, this is an exception to my normal rules though.

As to using FindSFML, as Hodgman says, I want a self contained project, so I build SFML as part of my solution. Different library versions, compiler flags etc are too painful later, I want to basically have a directory I can zip and drop on another machine and it just "works" without the user having to do a bunch of nonsense. At most, and only on Windows, I generally have to point folks at the VC redistributable to get that installed, other than that I want no other worries in distribution. This is an experience thing, back in the bad days I released a game and we were debugging oddball startup failures, game crashes etc for almost a month because of different libraries installed on different machines and all the hassles that involves. I wouldn't wish such an experience on anyone, especially the users.. :)

For inspiration:

http://www.boost.org/development/requirements.html#Directory_structure

https://developer.mozilla.org/en/docs/Source_code_directories_overview

https://developer.mozilla.org/en/docs/Mozilla_Source_Code_Directory_Structure

For more, you can also check out Item 2 from the book "Cross-Platform Development in C++"

// http://www.crossplatformbook.com/

@Hodgman,
@AllEightUp

Thank you for explaining and sharing your experiences. I can now understand why keeping the project self contained and not messing with the external libaries' layouts is beneficial in many ways. I'll put the SFML distro into my "external" directory as is and add it to my project's build process using CMake.

In order to do so I've already taken a deeper look into the tutorial project. Being the CMake beginner I am, I wasn't quite able to understand what was going on in the CMake scripts at first. After reading through the Cross Platform Test Driven Development Environment Using CMake articles, however, it was much easier. I haven't leveraged my newly acquired knowledge yet, but I'll do so in the next days, I hope. I'm looking forward to restructure other parts of my project with CMake for easier maintainability (e.g. making the framework a library to keep it separated from the gameplay stuff). smile.png

@Matt-D

Thanks! I have peeked into the book on Safari Books and put it on my ever growing list.

This topic is closed to new replies.

Advertisement