Jump to content

  • Log In with Google      Sign In   
  • Create Account


Like
13Likes
Dislike

Cross Platform Test Driven Development Environment Using CMake (Part 4)

By AllEightUp | Published Apr 10 2013 04:10 AM in General Programming
Peer Reviewed by (Josh Vega, Michael Tanczos, jbadams)

cross platform makefile c++ c++11 make cmake

In part 3 of the series, we finished up with the last target type and then added the desired unit testing library with a simple example of usage. The initial work was all about getting simple things to build and how to cover all the targets properly. It is time to get into some further bits of CMake and possible uses within your codebase. Solving some common coding problems, integrating better with IDE’s and cleaning up the CMake files themselves is what will be covered in this last part focused on CMake.

Parts 1 2 3 4 5

The final completed (within reason) environment: Attached File  WithSIMDAndiOS.zip   1.94MB   175 downloads


Being More Specific


One of the key things which I did in prior work was to use the simple generic detection variables in places where we needed to know something about the target. For instance, I used ‘APPLE’ to fix some of the early problems on OSX. What if you need to setup flags for a specific OS though? For instance, perhaps you have a network layer which has variations based on the target OS and specific versions of the OS? In a network system, you might have variations such as kqueue for BSD and derivatives (such as OSX), epoll for Linux and event ports for Windows. Additionally you may need to fall back to less scalable solutions based on the OS version, for instance if you use event cancellation in Windows event ports there are difficulties. You need to choose some other solution on Windows XP and prior versions due to bugs in the implementation. The need for more fine grained detection of platform/target OS is required to give the code more to work with when making such choices.

Much of the detection can be performed at compile time with the preprocessor. But is this good enough? First you have to figure out how to detect the compiler being run which can be difficult since finding preprocessor definitions is not always easy. This is even more complicated than it seems though due to compilers trying to be compatible with other compilers and impersonating them. For Windows compilers you may need to query if it defines ‘_MSVC’ and then check that it is not an imposter such as Intel Compiler or Vector C etc. Once the compiler is detected, what about the target version of the OS? The OS information is not often passed into the compiler and checking defines in various headers won't very often help. What we need is a more reasonable method of dealing with the detections, thankfully CMake supplies this for us.

Compiler Detection


CMake performs the specific detections we are looking for, in fact the generic ones we have used so far are derived from the detailed detections it has performed. In order to get the real compiler you can avoid the preprocessor difficulties by using the CMake defined variables: CMAKE_C_COMPILER_ID or CMAKE_CXX_COMPILER_ID. These two variables contain a string which corresponds to a specific compiler, even if that compiler were trying to impersonate another. In order to make detection more specific, you may use the following within a listfile:

IF( "${CMAKE_CXX_COMPILER_ID}" STREQUAL "MSVC" )
    SET( COMPILER_MSVC ON )
ELSEIF( "${CMAKE_CXX_COMPILER_ID}" STREQUAL "Intel" )
    SET( COMPILER_INTEL ON )
ELSEIF( "${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU" )
    SET( COMPILER_GNU ON )
ELSEIF( "${CMAKE_CXX_COMPILER_ID}" STREQUAL "Clang" )
    SET( COMPILER_CLANG ON )
ELSE()
    SET( COMPILER_UNKNOWN ON )
ENDIF()

With this script the specific compiler is identified correctly, impersonators are differentiated and if the user attempts to use an unknown compiler, say Vector C, then ‘COMPILER_UNKNOWN’ would be set. Before the user even bothers to compile, the CMake step could tell them that their setup is not supported. The user could decide to add the support, complain to the maintainer or of course give up.

OS Detection


Much like the compiler, there are variables to query information about the OS. There is only one variable in general “CMAKE_SYSTEM” which contains the full name and version information of the OS the build is being run on. The variable is helpfully broken into two parts by CMake so you can access the name and the version separately with: CMAKE_SYSTEM_NAME and CMAKE_SYSTEM_VERSION. Using the same pattern of code found in identifying the compiler, you can write detection for the specific OS names: Linux, FreeBSD, Windows and others. Also, dealing with OS versions is simple except we have to rely on CMake’s integer comparison abilities. Basically though, if you wanted to know if the Window’s version is higher than XP, you could use the following line:

IF( ${CMAKE_SYSTEM_VERSION} VERSION_MORE 5.1 )
    SET( OS_VERSION_NOT_XP ON )
ELSE()

CMake supplies a specialized comparison function which deals with version numbers such as ‘2.8.4.20130112’. This function performs the comparison by breaking the individual portions of the version string and doing the comparison on each piece separately.

CMake Generated Configuration Files


With all the new detection code, we need a better way to communicate the results to the codebase. Up until now 'ADD_DEFINITIONS' was good enough since we only required a couple definitions added to the code while compiling, with all the detection made possible by CMake though, we need a bulk transfer method. In order to supply bulk information to the codebase easily, CMake supplies a command which works somewhat like the Unix Sed command. It reads in a source file, searches for key text and replaces it with other text, then writes out a modified file to a given destination. The 'CONFIGURE_FILE' command is a rather poorly documented command which happens to supply a very nice and integrated solution of communicating with the codebase. While it only supplies two styles of replacement it is more than enough for nearly any type of configuration information you may desire.

It is simple to state that the command looks for ‘#cmakedefine’ and any text in the form of ‘@xxx@’ or ‘${xxx}’ but explaining what it does is far easier with a bit of example:

In a CMakeLists.txt file:

SET( THIS_IS_ON ON )
SET( THIS_IS_OFF OFF )

SET( TEST1 CameFromTest1 )
SET( TEST2 CameFromTest2 )

CONFIGURE_FILE( SourceFile.txt ResultFile.txt )

In SourceFile.txt:

#cmakedefine THIS_IS_ON
#cmakedefine THIS_IS_OFF

#define Test1Text "@TEST1@"
#define Test2Text "${TEST2}"

CMake will generate the following ‘ResultFile.txt’:

#define THIS_IS_ON
/* #define THIS_IS_OFF */

#define Test1Text "CameFromTest1"
#define Test2Text "CameFromTest2"

So, what is happening here is that as the command processes the input it looks up variables in the CMake environment to determine how the text replacement is made. In the first and second lines the ‘#cmakedefine’ is detected, CMake reads the variable name and looks it up. In the first case the variable is defined as a boolean true (‘ON’) and CMake outputs an normal preprocessor ‘#define’ with the variable name. (No value is included as you can see, just “defined”.) The second case is of course false (‘OFF’) and as such CMake outputs the definition as C style commented out. In the next two lines, the detected text is simply replaced with the contents of the CMake variable of the same name. Given these two styles of replacement, you can do just about anything you could ever require.

Take a look at the file 'Config.hpp.in' within the new build environment for a more complete example. The output is put in 'Libraries/Core/Include/Core/Config.hpp' when you generate the build environment so you can see an example of real usage.

Xcode Specialized Attributes


While we managed to fix the compilation process on Xcode, it was not as complete and proper as intended. Unfortunately I forgot to include the Xcode specific variables which notify it of the intended changes, so the IDE would sometimes override the settings in annoying ways. Xcode has a number of special purpose settings not found in some of the other IDE’s due to the specialized cross compilation mode it supports for iOS. In order to fix this, just add the following. The specific attributes are beyond the intended scope here but unless APPLE or CMake changes things, these should work just fine for our purposes of finishing off Xcode support properly. Also note that we look at the 'CMAKE_GENERATOR' variable and detect Xcode specifically instead of using the generic detections.

IF( CMAKE_GENERATOR STREQUAL Xcode )
    SET( CMAKE_XCODE_ATTRIBUTE_CLANG_CXX_LANGUAGE_STANDARD "c++11" )
    SET( CMAKE_XCODE_ATTRIBUTE_CLANG_CXX_LIBRARY "libc++" )
ENDIF()

GUI Options Revisited


In the third part of the series we added a single option which allowed disabling the unit tests. Options are pretty simple in the general case but eventually you will want to take more discreet control over certain things. In the future you may want to override target OS's instead of using the currently detected OS, you may want to setup CPU specific options etc. For example, we'll be adding SIMD code generation options and considering multiple CPU targets in the process.

The big deal with the options is that we want to display them in the CMake GUI but also remove options which don’t make sense. For instance, I wish to provide eventual support for multiple levels of Intel SSE and the option to target ARM Neon for iDevices and potentially native level Android. Of course, when compiling on ARM CPU's, enabling Intel SSE at any level makes no sense and of course the other way around for ARM Neon generation. We need a method of making the SSE and ARM mutually exclusive options. CMake does not directly support options being dependent on other options but thankfully, since this is a fairly common requirement, there is a macro with the functionality we desire already implemented. The macro we are looking for is called CMAKE_DEPENDENT_OPTION and it supplies what we desire in a fairly comprehensive, though potentially confusing, manner. As this is a helper, we need to tell CMake to load the file with the macro definition for this item. In the listfiles somewhere before you attempt to use the macro, add the following:

INCLUDE( CMakeDependentOption )

Because this file is supplied by CMake, you don’t need to tell CMake where to find it, and it will simply load as desired. It adds the macro with the following call signature:

CMAKE_DEPENDENT_OPTION( <option name="name">
	<string description="description">
    <default on/off="on/off">
    <predicate>
    <on/off if="if" predicate="predicate" is="is" false="false">
)

The first three arguments are the same as the OPTION command, while the next two options are what determine if the option will be shown or not and what value it will take when hidden. So, an example using the intended SSE options:

OPTION( INTEL_SSE "Enable SSE on Intel CPU" ON )
CMAKE_DEPENDENT_OPTION( INTEL_SSE_2 "Enable SSE 2 instructions." OFF "INTEL_SSE" OFF )

So, in the CMake GUI, SSE will be shown and enabled by default. SSE 2 will be shown in a disabled state by default. If you turn off SSE 1 and regenerate, the SSE 2 option will disappear and be set to off. A nice benefit though, if you had set SSE2 to on and then later re-enable SSE1, it would remember the last setting. Note that the predicate in this case is quoted, this is not required for the simple case here but if you want “INTEL_SSE AND Other” the quotes tell CMake to process the entire boolean evaluation as the argument instead of passing the pieces as individuals. Given this new tool, you can see how it will allow us to select cross compilation targets such as iPhone or Android and show only valid options for those targets. The new environment shows how to write a fairly complete version of options using this new ability to prevent incorrect or contradictory options from being shown. (NOTE: CMake GUI does not update the options until you regenerate. So, in order to enable SSE 4 or AVX, you have to enable the next higher version of SSE, regenerate to get the next one to show up, repeat.)

Reorganizing And Cleaning


Eventually, as you add more abilities and target support, your listfiles will become unmanageable, this is quite easy to fix. The first thing to do is add a new directory at the base of the project, we will call it “CMake” as a nice original name. We will be moving the individual portions of CMake code into files under the new sub-directory in order to organize related tasks and get them out of the root listfile.

Reorganization


The first thing to do is decide on a naming convention. For our purposes we will simply follow the other items in the project and use upper case camel case name. The files we create will have the extension ‘.cmake’ which is a fairly common convention. With those decisions out of the way, we’ll start cleaning things up. Create a file under the new directory named “Setup.cmake”. The ‘INCLUDE’ for the dependent options is a general thing we will be using, so move that to the setup file and replace it in the main file with "INCLUDE( CMake/Setup.cmake )". In the future, certain other includes will be needed and we'll add to this file as we go.

After we get done adding all the SIMD options and cross compilation targets, the list of options is a pretty large chunk of code along with all the compiler management sections. So, those are good sections to move into other files. Move all the options for your build over to ‘CMake/Options.cmake’ and add an include to the new file in the root listfile. (After the PROJECT command or required variables and GUI changes won’t be available.) If you keep doing this for other sections, you will eventually end up with a listfile which is nothing more than a ‘PROJECT’ command and a bunch of inclusions or added directories. In the supplied environment, you can see the cleaned up listfile's and where the different sections were moved.

IDE Integration: Target Organization


Once your codebase progresses, it is common to end up with 20 or more individual targets in the IDE's. Left as a flat list, finding things in the IDE can become difficult and annoying. It is time to clean that up by grouping things together and reorganizing the project. The abilities used here have no effect on actual building and are purely intended for the IDE's. The first thing we will do is move all the unit test targets into a grouping called "Tests".

CMake does not supply a specific command to create groupings within the project but it does support the concept. What we need is to use the CMake properties system to tell it where to place various items. Properties in general can be used for many things but as there are a lot of them, we only cover one specifically here. The specific property is attached to the targets we generate, for instance the '_TestMath' target is what we will use as an example. Add the following to the listfile which contains the unit test, within the 'IF( BUILD_UNITTESTS )' section and after we define the target:

SET_PROPERTY( TARGET _TestMath PROPERTY FOLDER Tests )

Now when you regenerate the project Xcode, Visual Studio and the other IDE's will create a non-target folder called "Tests" and place this unit test target within the folder. As you move forward and add more unit tests, they will all be organized under this new folder.

IDE Integration: Header/Source Organization


CMake also allows organization of individual files within the targets using the 'SOURCE_GROUP' command. If for instance you wished to put all headers from the math library under the group "Math Includes", you could add the following to your math listfile:

SOURCE_GROUP( "Math Includes" FILES ${INCLUDE_FILES} )

Eventually as the math library fleshes out you might want to subdivide various headers into nicely named embedded folders. Let's say I have a group of files which deal with intersections of various types and I want them under "Math Includes/Intersection" within the IDE. You can use the following

SOURCE_GROUP( "Math Includes&#092;&#092;Intersection" FILES ${INTERSECTION_INCLUDE_FILES} )

Note:  
The ampersand items are supposed to be the backslash character: '\'. The article posting process keeps replacing them with the HTML representations.


Using this command, you can organize files within IDE's in a nice manner which makes navigating your code easier.

Cross Compilation


Cross compilation is not a difficult concept but can be a little confusing to get setup. We will be walking through the addition of an iOS target option to the project. The first thing to do is ‘know you target’ which is fairly simple in this case, an ARM processor and on later units the option to use ARM Neon (SIMD) instructions to speed things up. Since Intel SSE is not supported, we need to make sure targeting ARM disables all SSE settings and enables the ARM Neon instruction set as an option. If you followed along and understood the dependent argument discussion this is fairly easy.

We start by adding an option for the new target: 'TARGET_IOS’ which displays the new target. There is a problem though, this command is only going to be valid when generating Xcode projects or Unix Makefiles when running on OS X. To Keep things simple, we only support iOS under Xcode and leave the makefile generation of iOS up to the reader as an exercise at this point. The following will limit this option to Xcode generators only:

IF( CMAKE_GENERATOR STREQUAL Xcode )
OPTION( TARGET_IOS "Target iOS" OFF )
ENDIF()

Next we need to hide the SSE options if iOS is targeted, so change the SIMD_SSE option into a dependent as follows:

CMAKE_DEPENDENT_OPTION( SIMD_SSE "Enable SSE" ON "NOT TARGET_IOS" OFF )

Now if you regenerate on an OS X machine, you get the new target type listed in the CMake GUI and if you enable it then regenerate the SSE options would disappear.

At this point, we simply need to tell CMake to output a project which is specifically designed to build iOS applications. Hopefully the usage of CMake in a general manner has become second nature and as such, we don’t need to go over details of how to change compiler flags. There are only a couple steps to getting this to work, the first part just informs CMake of the changed target and then dealing with some of the target features:

SET( CMAKE_CROSSCOMPILING TRUE )
SET( CMAKE_SYSTEM_NAME "Darwin" )
SET( CMAKE_SYSTEM_PROCESSOR "arm" )

The required lines are fairly self documenting and tell CMake that we have a custom (not this system) target in mind, which OS and CPU to target. Things get a bit more complicated in the remaining items because they are all OS X/Xcode specific and not as self documenting. We need to change the SDK being used to compile from the OS X SDK’s to the iOS SDK’s, change target cpu’s and bit sizes and then tell Xcode to change it’s behavior to be iOS specific. Additionally, since iOS doesn’t support console targets, we have to disable the unit tests since they would fail to compile. Adding the following:

SET( SDK_VERSION "6.1" )
SET( SDK_LOCATION "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer" )
SET( ACTIVE_SDK "${SDK_LOCATION}/SDKs/iPhonOS${SDK_VERSION}.sdk" )
SET( CMAKE_OSX_SYSROOT "${ACTIVE_SDK}" )

SET( CMAKE_OSX_ARCHITECTURES "${ARCHS_STANDARD_32_BIT)" )
SET( CMAKE_XCODE_EFFECTIVE_PLATFORMS "-iphoneos;-iphonesimulator" )

SET( BUILD_UNIT_TESTS OFF )

So, the first section of code changes the SDK to an iPhone SDK, 6.1 in this case. The code is written such that you could target older SDK’s but new submissions to the App Store require the latest SDK. The second section tells Xcode to target the iPhone, the specifics are beyond the intended scope here, though this will provide a working environment now and likely in the future unless Apple decides to move things around again. And finally, we disable the console commands for unit tests since iOS has no concept of a console application.

At this point, if you enable TARGET_IOS and regenerate, load the result into Xcode and you will see the iPhone/Simulator targets and be able to compile. (NOTE: This does not provide the App Signing certificate, if installed the certificate correctly, that should happen automatically, but idevice target will not build until you get the certificate setup correctly.)

Conclusion


While not as detailed as prior parts of this series, the information covered fleshes out the knowledge required to create a multi-platform and cross compilation capable CMake build environment. Further articles will transition to using this environment and extending it further but other than small things, will not be covering further detail of CMake. As a practical guide, the presented material supplemented with some Google searching (GameDev.net and StackOverflow being a very common source of answers), should give you all the tools required to move forward compiling on nearly any target.



License


GDOL (Gamedev.net Open License)




Comments

This series is pure gold (or bitcoins;)). Please keep them coming.

Getting 2 errors on VC2008 SP1:

 

1) Error 1 fatal error C1083: Cannot open include file: 'stdint.h': No such file or directory d:\devel\withsimdandios\libraries\core\include\core\Config.hpp 15 Math

 
2) Error 2 error C2908: explicit specialization; 'std::tr1::tuple<>' has already been instantiated D:\devel\WithSIMDAndiOS\External\gmock-1.6.0\fused-src\gtest\gtest.h 736 _TestCore
 
which is described here:
 
 
solution appears to be:
 
#  define GTEST_USE_OWN_TR1_TUPLE 0

Getting 2 errors on VC2008 SP1:

 

1) Error 1 fatal error C1083: Cannot open include file: 'stdint.h': No such file or directory d:\devel\withsimdandios\libraries\core\include\core\Config.hpp 15 Math

 
2) Error 2 error C2908: explicit specialization; 'std::tr1::tuple<>' has already been instantiated D:\devel\WithSIMDAndiOS\External\gmock-1.6.0\fused-src\gtest\gtest.h 736 _TestCore
 
which is described here:
 
 
solution appears to be:
 
#  define GTEST_USE_OWN_TR1_TUPLE 0

Err, sorry.  You need to look at part one where I specify that this is a C++11 specific series and requires VC2012.  I'm sure I could make it work for 2008 but that was not the purpose unfortunately.

 

I realize as a build environment it could, probably should, not be compiler specific, but the future work is all about C++11.

OK, it was about time I upgraded my VC. Now all works fine in 2012. Do you plan more articles in this series?

 

Do you have a CMake-based solution for external libs like zlib and libpng? Which OpenGL windowing lib/framework (I assume you are using OGL to be portable) are you using for your projects?

 

Awesome series! Thanks again!

I'm about to start writing the article to go with an implementation of SSE1-AVX + Arm Neon SIMD math library.  The intention is not to teach SIMD but a practical method of implementing scalable and cross compiling code in this CPU specific area.  Other than tweaking the build environment a bit to support the requirements though, I'm pretty done with CMake specifics for the time being.

 

It is not ready for primetime but you can check out: https://code.google.com/p/a8u-tutorial-sources which has some of the experiments going on.  The OpenGL stuff can be found in the WithSupportLibs directory, I was just puttering around there so don't expect anything polished, just SFML and Horde3d integrated and basically ready to use.  (Takes a bit of manual file copying to run some things at the moment though, haven't cleaned it up.)

 

Anyway, the intention is to move on to the math library setup, I'm a nitpicker and math is the glue to making a game so I find it critical to have a decent library from the start.  Hence, that's the first step on the way to writing a series towards an actual game project.

I just found this great VC plugin for gtest integration: http://visualstudiogallery.msdn.microsoft.com/f00c0f72-ac71-4c80-bf8b-6fe381548031/view/Discussions

 

solves my issue with cmake-generated solution not playing as nicely with VS as handmade one (with custom post build step).

 

It does not work perfectly when working with cmake (plugin only looks for .pdb files in default location atm) but is still pretty great.

 

btw. just-released cmake 2.8.11 supports toolset selection for VC and XCode so it is now possible to create applications targeting win XP with VC2012.


Note: Please offer only positive, constructive comments - we are looking to promote a positive atmosphere where collaboration is valued above all else.




PARTNERS