• # Cross Platform Test Driven Development Environment Using CMake (Part 3)

General and Gameplay Programming

• Posted By Guest
In the last part I showed how to start adding static libraries and setup to share information between the different portions of the CMake build system. This part will cover the last of the build targets, shared libraries which are a little more difficult than the static libraries. The difficulty is not on the CMake side but instead due to Windows being a bit of a pain in this area and requiring some additional work. Once the shared libraries are complete though, the final portion of the environment will be added, the unit test library. Unit tests are a great benefit to nearly any project. A set of proper tests can help in many ways, the most obvious being to catch bugs early. Unfortunately a number of the test libraries end up requiring a lot of boilerplate code and become a pain to write. As such, when looking for a testing library the number one goal was to find something as minimal and non-intrusive as possible. At the same time, a mocking system is also useful to have for larger and more complicated testing purposes. The Google Mock library googlemock supplies both the mocking and unit test libraries and it is very simple to use. Download and uncompress it somewhere if you don't have it from Part 2 of the series. Parts 1 2 3 4 5

# Shared Libraries

## Controlling Tests

The last thing to be done is to control if unit tests are compiled and built. When you start working this is not a big deal of course as you will likely only have a couple libraries and tests. After a while though, you may end up with more little "test" targets than actual libraries and applications. So, it is a good idea to go ahead and add the ability to disable tests from day one. Add the following item: CMakeEnvironment/CMakeLists.txt:  # ############################################### # Allow unit tests to be disabled via command # line or the CMake GUI. OPTION( BUILD_UNIT_TESTS "Build unit tests." ON )  Add this just under the 'PROJECT' command and it will show up in the CMake GUI as something you can turn on and off.

# Putting The Results To Use

Two and a half articles later, we are finally ready to put our work to good use. I'm going to start by writing a 3D vector class since it is a common item and easily unit tested. The first thing to do is of course get the completed environment. This file includes the googlemock library so it's quite a bit larger than the prior items:

## The Math Library

I'm going to keep this very simple for the purposes of this article. I will not be implementing the actual class, just enough to show a unit test in action. Future articles will likely build off this work but this is going to close out the CMake work for the time being so we want to keep things simple. Let's start by adding the directories for the new 'Math' library:  CMakeEnvironment Libraries Math Include Math Source Tests  Ok, so the first question is likely to be why did I duplicate the directory 'Math' under the 'Include' directory? This is simply a polution prevention item. Let's say you have a custom 'Collections' library and within that there is a 'Vector.hpp' file and of course the math library could have a 'Vector.hpp' file. If you include without the prefix of 'Collections' or 'Math', which file are you including? With the way we will setup the libraries, this problem is solved by forcing the users of the libraries to qualify the includes as follows:  #include  This is another one of the personal preference items but I prefer avoiding problems from day one and not simply assuming they won't happen. If you don't like this, once we get done feel free to remove the nested directory. But be warned that in future articles you'll see some good reasons to use this pattern which can greatly simplify normally difficult processes. Let's fill in the hierarchy with files now, add the following: CMakeEnvironment/Libraries/Math/CMakeLists.txt:  SET( INCLUDE_DIRS ${CMAKE_CURRENT_LIST_DIR}/Include ) INCLUDE_DIRECTORIES(${INCLUDE_DIRS} ) SET( INCLUDE_FILES Include/Math/Math.hpp Include/Math/Vector3.hpp ) SET( SOURCE_FILES Source/Vector3.cpp ) ADD_DEFINITIONS( -DBUILDING_WORLD ) ADD_LIBRARY( Math STATIC ${INCLUDE_FILES}${SOURCE_FILES} ) # Export the include directory. SET( MATH_INCLUDE_DIRS ${INCLUDE_DIRS} PARENT_SCOPE )  Add empty files for: Include/Math/Math.hpp Include/Math/Vector3.hpp Source/Vector3.cpp Tests/Main.cpp Regenerate the environment and you should have your new Math library. ## The First Unit Test The first thing we want to do is add a unit test holder before we add any code to the library. What we need is a simple executable which can include the files from the library. Since this is part of the math library, we can simply add the test within 'CMakeEnvironment/Libraries/Math/CMakeLists.txt'. Additionally, we want the test to honor the flag we setup such that the test is not included when tests are disabled. Add the following to the end of the Math libraries CMake file:  # Make a unit test holder for the Math library. IF( BUILD_UNIT_TESTS ) # Add the gmock include directories. INCLUDE_DIRECTORIES(${GMOCK_INCLUDE_DIRS} ) ADD_EXECUTABLE( _TestMath Tests/Main.cpp ) TARGET_LINK_LIBRARIES( _TestMath Math GMock ${CMAKE_THREAD_LIBS_INIT} ) ADD_TEST( NAME _TestMath COMMAND _TestMath ) ENDIF( BUILD_UNIT_TESTS )  Regenerate and you get a new target '_TestMath'. In the CMake GUI if you set 'BUILD_UNIT_TESTS' to 'FALSE' and regenerate, the new target goes away as we desired. But, there is actually one more thing we want to do to make this even better in the future. In the root CMake file with the 'OPTION' command defining the 'BUILD_UNIT_TESTS' variable, add the following bit right afterward:  # ############################################### # Enable the CMake built in CTest system if unit # tests are enabled. IF( BUILD_UNIT_TESTS ) ENABLE_TESTING() ENDIF( BUILD_UNIT_TESTS )  Make sure to set 'BUILD_UNIT_TESTS' back to 'TRUE' and regenerate. A new target shows up in the build: "RUN_TESTS". This is a nice little target when you have many more tests then we will have here. Basically if you attempt to build this target is runs all the executables you add with 'ADD_TEST'. It is also great for automated build/continuous integration environments since running the target is easy compared to finding all the individual tests. Now, lets add the actual unit test code: CMakeEnvironment/Libraries/Math/Tests/Main.cpp  #include int main( int argc, char** argv ) { ::testing::InitGoogleTest( &argc, argv ); return RUN_ALL_TESTS(); }  This is a do nothing bit of code right now until we add some code to the Math library and of course add some tests. So, let's fill in the vector class real quick: CMakeEnvironment/Libraries/Math/Include/Math/Vector3.hpp:  #pragma once #include  CMakeEnvironment/Libraries/Math/Source/Vector3.cpp:  #include  Yes, the default constructor does not initialize to zero's. That is a topic for another article. Rebuild and the library should build, the test case should build and even run with the following output:  [==========] Running 0 tests from 0 test cases. [==========] 0 tests from 0 test cases ran. (1 ms total) [ PASSED ] 0 tests.  So it is time to add our first tests. Add the following files: CMakeEnvironment/Libraries/Math/Tests/TestAll.hpp:  #include  CMakeEnvironment/Libraries/Math/Tests/TestConstruction.hpp:  TEST( Math, Vector3f ) { Math::Vector3f test0( 0.0f, 0.0f, 0.0f ); EXPECT_EQ( 0.0f, test0.X() ); EXPECT_EQ( 0.0f, test0.Y() ); EXPECT_EQ( 0.0f, test0.Z() ); Math::Vector3f test1x( 1.0f, 0.0f, 0.0f ); EXPECT_EQ( 1.0f, test0.X() ); EXPECT_EQ( 0.0f, test0.Y() ); EXPECT_EQ( 0.0f, test0.Z() ); Math::Vector3f test1y( 0.0f, 1.0f, 0.0f ); EXPECT_EQ( 0.0f, test0.X() ); EXPECT_EQ( 1.0f, test0.Y() ); EXPECT_EQ( 0.0f, test0.Z() ); Math::Vector3f test1z( 0.0f, 0.0f, 1.0f ); EXPECT_EQ( 0.0f, test0.X() ); EXPECT_EQ( 0.0f, test0.Y() ); EXPECT_EQ( 1.0f, test0.Z() ); }  Modify CMakeEnvironment/Libraries/Math/CMakeLists.txt by adding the test headers:  ADD_EXECUTABLE( _TestMath Tests/Main.cpp Tests/TestAll.hpp Tests/TestConstruction.hpp )  And include the 'TestAll.hpp' from the file CMakeEnvironment/Libraries/Math/Tests/Main.cpp:  #include #include "TestAll.hpp" int main( int argc, char** argv ) { ::testing::InitGoogleTest( &argc, argv ); return RUN_ALL_TESTS(); }  Regenerate, build and run. The test should output the following:  [==========] Running 1 test from 1 test case. [----------] Global test environment set-up. [----------] 1 test from Math [ RUN ] Math.Vector3f [ OK ] Math.Vector3f (0 ms) [----------] 1 test from Math (1 ms total) [----------] Global test environment tear-down [==========] 1 test from 1 test case ran. (5 ms total) [ PASSED ] 1 test.  Congratulations, you have a unit test for your new Vector3 class. The completed environment and example unit test can be downloaded here Please provide feedback on any build problems you might have and I'll attempt to get things fixed. # Conclusion In this article we covered dealing with OS specific difficulties involving Windows and a couple more OsX issues. It has been a long haul to get to this point dealing with nitpicky items and a lot of explanation. The final result is a fairly easy to use and maintain environment which you can use to build unit tests and apply test driven development features to your coding. In a future article I intend to expand on the Math library and show some of the big utilities of using an environment such as this day to day. For now, my fingers are tired and my own code calls to me. Report Article ## User Feedback Last few weeks I am moving from Win-only (Visual Studio) build to portable CMake version. One of few things to still figure out is unit testing integrated into the process. I came back after Easter and see this series - best gift I could ask for :) Great work and thanks for doing this! #### Share this comment ##### Link to comment ##### Share on other sites I might be missing something but: 1) should gmock/gtest results be displayed in VS "Output" window during the build? 2) breaking a test (say: Math::Vector3f test0( 0.0f, 0.0f, 0.0f ); EXPECT_EQ( 1.0f, test0.X() ); ) does not break the build and basically all looks OK. Is that desired behavior? #### Share this comment ##### Link to comment ##### Share on other sites Since you mentioned ADD_DEFINITIONS...: how do you deal with #defines that have to be added only in specific variant of the build (like debug)? #### Share this comment ##### Link to comment ##### Share on other sites I might be missing something but: 1) should gmock/gtest results be displayed in VS "Output" window during the build? 2) breaking a test (say: Math::Vector3f test0( 0.0f, 0.0f, 0.0f ); EXPECT_EQ( 1.0f, test0.X() ); ) does not break the build and basically all looks OK. Is that desired behavior? The default behavior for GMockGTest is to dump the test output to std::cout and it doesn't know about OutputDebugString. I was considering adding an example of how to do that but this article was already quite large so skipped it. I should have probably mentioned that the way I usually work in Windows is to just set a break point at the end of the main, not a great solution but you can tab back to the output and see if the tests completed and all passed. For #2, GMock/GTest catch and eat all errors, even full on program crashes are usually caught correctly and simply output as a failed test, which is proper. :) #### Share this comment ##### Link to comment ##### Share on other sites Since you mentioned ADD_DEFINITIONS...: how do you deal with #defines that have to be added only in specific variant of the build (like debug)? This one gets to be a bit more difficult. I was considering if I should cover this later as at this point you get into some of the less well understood areas of CMake. But, there is a relatively simple solution if you need per build type definitions. CMake will add "_DEBUG" flags in all of the generators. So, if you need to differentiate some flags, just "#if _DEBUG" for the debug versions "#else" for the release versions. #### Share this comment ##### Link to comment ##### Share on other sites I might be missing something but: 1) should gmock/gtest results be displayed in VS "Output" window during the build? 2) breaking a test (say: Math::Vector3f test0( 0.0f, 0.0f, 0.0f ); EXPECT_EQ( 1.0f, test0.X() ); ) does not break the build and basically all looks OK. Is that desired behavior? The default behavior for GMockGTest is to dump the test output to std::cout and it doesn't know about OutputDebugString. I was considering adding an example of how to do that but this article was already quite large so skipped it. I should have probably mentioned that the way I usually work in Windows is to just set a break point at the end of the main, not a great solution but you can tab back to the output and see if the tests completed and all passed. For #2, GMock/GTest catch and eat all errors, even full on program crashes are usually caught correctly and simply output as a failed test, which is proper. Actually that is not what I meant. ATM my VS build is configured in this pretty sweet way: http://leefrancis.org/2010/11/17/google-test-gtest-setup-with-microsoft-visual-studio-2008-c/ gtest_main executable is executed as post-build step of dependent target which most importantly causes fail of main build (tests binary returns non-zero to VS) on failed test. On top of that, since testing is now part of the build, every test result is displayed in "Output" window and on error you can just double-click in it and it jumps to failed ASSERT/EXPECT (like wit normal errors and warnings). #### Share this comment ##### Link to comment ##### Share on other sites I just love articles that explain a single problem in much detail, especially when it comes to the most basic thing before you start developing: the environment itself. Combined with the TDD approach, I'll definitely be using this in my own project. :) One little thing: could you please correct the navigation links? The link to part 2 points to part 1. ;) #### Share this comment ##### Link to comment ##### Share on other sites I might be missing something but: 1) should gmock/gtest results be displayed in VS "Output" window during the build? 2) breaking a test (say: Math::Vector3f test0( 0.0f, 0.0f, 0.0f ); EXPECT_EQ( 1.0f, test0.X() ); ) does not break the build and basically all looks OK. Is that desired behavior? The default behavior for GMockGTest is to dump the test output to std::cout and it doesn't know about OutputDebugString. I was considering adding an example of how to do that but this article was already quite large so skipped it. I should have probably mentioned that the way I usually work in Windows is to just set a break point at the end of the main, not a great solution but you can tab back to the output and see if the tests completed and all passed. For #2, GMock/GTest catch and eat all errors, even full on program crashes are usually caught correctly and simply output as a failed test, which is proper. Actually that is not what I meant. ATM my VS build is configured in this pretty sweet way: http://leefrancis.org/2010/11/17/google-test-gtest-setup-with-microsoft-visual-studio-2008-c/ gtest_main executable is executed as post-build step of dependent target which most importantly causes fail of main build (tests binary returns non-zero to VS) on failed test. On top of that, since testing is now part of the build, every test result is displayed in "Output" window and on error you can just double-click in it and it jumps to failed ASSERT/EXPECT (like wit normal errors and warnings). I took a look at the description you mentioned. That seems like a nice integration. I'll look into it further. The one thing about the integration though is I will often have a couple tests which fail as I do refactoring work, so I don't want the tests to cause a failure when I run things most of the time. I just want to know which ones failed as I make changes. Especially when you have say 100+ tests going, a single failure may not be a horrible problem though knowing about it is a good idea. #### Share this comment ##### Link to comment ##### Share on other sites I just love articles that explain a single problem in much detail, especially when it comes to the most basic thing before you start developing: the environment itself. Combined with the TDD approach, I'll definitely be using this in my own project. One little thing: could you please correct the navigation links? The link to part 2 points to part 1. ;) Err, oops.. :) I'm traveling at the moment but I'll try to get it fixed up. Thanks for the feedback though, glad you like it. As a note, I haven't yet decided what to do with the fourth article. The original intention was to get to an x-plat SIMD capable math library and use that to show some of the useful bits of the setup and why I use the embedded extra named directory. I'm trying to decide if I should continue on with the CMake first though as the setup and such has updated quite a lot. There are some things in the posted one done in less than the best manners, functional yes but intended for simplicity instead of using the absolute best practices. (With CMake it is not always obvious what the best case is of course. :)) I may attach the more advanced version up as a little after note to part 3 but the features will take another article to describe. Some of the interesting bits for folks would likely be the use of a configure system to generate a "Config.hpp" specific to your target and settings. The new settings include IOS support, control over SIMD (SSE1-AVX for Intel and Neon for Arm) and general cleanup and moving things around. Also of note was that In the posted version I forgot to properly cover the Xcode specific attribute variables which means Xcode can sometimes be a bit annoying since some things may fail once in a while. (It's just a random "I don't support Cxx11' compiles, the next time it will likely work.) Anyway, that is a bunch more stuff which could take another part added to the article. Probably a good idea to go ahead and do that and push the other stuff a bit later. #### Share this comment ##### Link to comment ##### Share on other sites I might be missing something but: 1) should gmock/gtest results be displayed in VS "Output" window during the build? 2) breaking a test (say: Math::Vector3f test0( 0.0f, 0.0f, 0.0f ); EXPECT_EQ( 1.0f, test0.X() ); ) does not break the build and basically all looks OK. Is that desired behavior? The default behavior for GMockGTest is to dump the test output to std::cout and it doesn't know about OutputDebugString. I was considering adding an example of how to do that but this article was already quite large so skipped it. I should have probably mentioned that the way I usually work in Windows is to just set a break point at the end of the main, not a great solution but you can tab back to the output and see if the tests completed and all passed. For #2, GMock/GTest catch and eat all errors, even full on program crashes are usually caught correctly and simply output as a failed test, which is proper. Actually that is not what I meant. ATM my VS build is configured in this pretty sweet way: http://leefrancis.org/2010/11/17/google-test-gtest-setup-with-microsoft-visual-studio-2008-c/ gtest_main executable is executed as post-build step of dependent target which most importantly causes fail of main build (tests binary returns non-zero to VS) on failed test. On top of that, since testing is now part of the build, every test result is displayed in "Output" window and on error you can just double-click in it and it jumps to failed ASSERT/EXPECT (like wit normal errors and warnings). If you add executing the individual test as a post-build step, the test will be run as a part of the build (and rerun every time the test is recompiled due to changes). # Make a unit test holder for the Math library. IF( BUILD_UNIT_TESTS ) # Add the gmock include directories. INCLUDE_DIRECTORIES(${GMOCK_INCLUDE_DIRS} )

Tests/Main.cpp
)
Math
GMock

##### Share on other sites

hi Abi,

First item:   Oops!  :)  Think I fixed it later but didn't update the various zip file attachments.  GameDev needs a better method of integrating code into articles as the zip files are a pain to keep maintained.

Second:  I'm not sure I'm following the idea.  Though I suspect perhaps you are saying something like:

a .cmake file in the projects/libraries etc directories we include and each of those then defines what to include/use so we are only at one step removed?  I might have to play with this, but honestly, I'm not sure it really changes the problem, just moves it.  I'll have to think about it, I still have a couple more articles going so perhaps I'll hit a CMake part 5 with suggestions and fixes included. :)

##### Share on other sites

[...]

Second:  I'm not sure I'm following the idea.  Though I suspect perhaps you are saying something like:

a .cmake file in the projects/libraries etc directories we include and each of those then defines what to include/use so we are only at one step removed?  I might have to play with this, but honestly, I'm not sure it really changes the problem, just moves it.  I'll have to think about it, I still have a couple more articles going so perhaps I'll hit a CMake part 5 with suggestions and fixes included.

Well, I'm just saying that after a PROJECT (<MYLIBNAME>) directive, it seems like cmake makes the <MYLIBNAME>_SOURCE_DIR variable defined to point where sources for that project are, so if <MYLIBNAME>'s includes are in a well defined relative path from its sources, you can get a valid pointer to their location everywhere by just using such variable e.g. <MYLIBNAME>_SOURCE_DIR/../Includes if there's an "Includes" folder at the same level where a Sources folder is.

Hope this clarifies my idea a bit, and looking forward parts >=5 for additional great content!

##### Share on other sites

[...]

Second:  I'm not sure I'm following the idea.  Though I suspect perhaps you are saying something like:

Well, I'm just saying that after a PROJECT (<MYLIBNAME>) directive, it seems like cmake makes the <MYLIBNAME>_SOURCE_DIR variable defined to point where sources for that project are, so if <MYLIBNAME>'s includes are in a well defined relative path from its sources, you can get a valid pointer to their location everywhere by just using such variable e.g. <MYLIBNAME>_SOURCE_DIR/../Includes if there's an "Includes" folder at the same level where a Sources folder is.

Hope this clarifies my idea a bit, and looking forward parts >=5 for additional great content!

Think I understand now.  Unfortunately it leaves a couple problems to be solved.  There are three things which need to be exported from the libraries:

:The library to link against, and any which it may need to add to the build.  For instance, if I link  in a network library, I'd also need to pull ws_32.dll on windows to make it function.

:The various extra defines for the library, sometimes you have some extra's required, again picking on Window's you might need to define an OS version.

:The include directory of course but it can actually be multiple directories.  Say for instance you have a renderer, it may need to include D3D/OpenGL directories also.

I'll keep looking at this but for the time being I think I like the explicit nature of the current version.  It is a bit verbose but that let's it adapt to any edge cases.  I'll keep thinking about it though.

##### Share on other sites
Hi AllEightUp,
I hit the same problem as you did with TR1 tuple lib in Xcode. Apparently google guys already saw that coming and added few compiler flags.

Just add one of the following to root CMakeLists.txt
# To force Google Test to use its own tuple library
# or if you don't want Google Test to use tuple at all

Hope that helps.

##### Share on other sites

I had tried to use the various flags, such as that one, but never got a working build for some reason.  I decided that a little cheatery was just as easy as figuring out the problems.  I will give it another try at some point but I was hoping the library would be updated before I tried again.

## Create an account

Register a new account