# C++ testing frameworks

This topic is 3340 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

What are some good C++ testing frameworks ? My biggest problem in my last major C++ project was finding exactly where bugs were coming from, and not breaking old code when I introduce new features. This time around I want to be a lot more methodical and a proper testing framework is a good start I think.

##### Share on other sites

Personally I took CppUnitLite and heavily modified it so it was suitable for my purposes. It's working out really well since I get to tune it exactly for my needs, but if you've not got the time then check out the above links and see which features you find most important.

##### Share on other sites
I've been using Boost.Test for a while now, and that works really well too.

##### Share on other sites
I'm using Cpp Unit. Working really well.

##### Share on other sites
Google has one. I think it's fairly new. I've never used it but it looks pretty nice.

On my current project I just rolled my own. I don't think it's that crazy to roll your own, since it doesn't take that much code, and you might want to adapt your testing strategy to the needs of your project.

##### Share on other sites
Quote:
 Original post by OrangyTangPersonally I took CppUnitLite and heavily modified it so it was suitable for my purposes. It's working out really well since I get to tune it exactly for my needs, but if you've not got the time then check out the above links and see which features you find most important.
Same here, though I found the modifications needed to be pretty minor. CppUnit is nice but a little heavy to apply, and Boost.Test... my goodness. I'm pretty sure unit tests aren't meant to be that painful.

##### Share on other sites
I've been using UnitTest++ recently.

##### Share on other sites
Quote:
 Original post by SneftelBoost.Test... my goodness. I'm pretty sure unit tests aren't meant to be that painful.

Heh, how do you mean? I haven't yet used it on any large projects, but so far, it's been surprisingly easy to use. Am I just missing something major here? [grin]

##### Share on other sites
Quote:
 Original post by SpoonbenderHeh, how do you mean? I haven't yet used it on any large projects, but so far, it's been surprisingly easy to use. Am I just missing something major here? [grin]
Perhaps it's just a matter of personal preference. At a minimum, I feel a testing framework should free you from having to do accounting for your own tests. I'd like to write a class, and write tests for it inline, easily and without any repetition or extra boilerplate or files. That's the only difficult part of writing such a framework, after all... the rest pretty much comes down to a big AND clause. Boost.Test makes me put each unit test in a separate class, written out in longhand, manually added to a test suite. I consider that to be too much work. Surely the organization that came up with Boost.Spirit and Boost.Lambda can use a few Stupid C++ Tricks to make that stuff more streamlined?

##### Share on other sites
You do know that manually registering your tests and all that is optional, right? They have a bunch of macros available for automatically registering tests, and even for organizing them in test suites.

My tests look roughly like this:

#include <boost/test/unit_test.hpp>#define BOOST_TEST_MAINBOOST_AUTO_TEST_CASE( foo_test ){ BOOST_CHECK(2+2 == 4);}BOOST_AUTO_TEST_CASE( bar_test){ BOOST_CHECK_NO_THROW(dostuff());}

And if you want to use a fixture in one of your tests:
struct fixture_class {  fixture_class() : some_member(42) {    // perform set-up for the test  }  ~fixture_class() {    // perform any necessary tear-down after the test  }  int some_member;};BOOST_FIXTURE_TEST_CASE( test_name , fixture_class ){  BOOST_CHECK(some_member == 42);}

I agree, it becomes a major pain if you start fiddling with manually registering your tests in test suites and all that. But as long as you stick with the automatically-registered ones, it doesn't require a lot of work to use. (Then again, I haven't used CppUnitLite, so can't really compare).

Not trying to "convert" anyone though, just curious because if it turns out other frameworks are easier still, I might as well use one of those instead. [grin]

##### Share on other sites
Quote:
 Original post by jpetrieI've been using UnitTest++ recently.

I'm a UnitTest++ -user too. At my company we evaluated a number of unit test suites and this one had the best combination of features and ease-of-use at that time. Some niggles: the lack of unicode support, as well as the difficulty in selecting specific tests to run.

I also liked CxxTest which uses Python (or Perl) to pre-process the test files. Easy-to-use, but with a lot of setup requirements that made it less suitable to the development environment here.

##### Share on other sites
Quote:
 Original post by SpoonbenderAnd if you want to use a fixture in one of your tests:*** Source Snippet Removed ***I agree, it becomes a major pain if you start fiddling with manually registering your tests in test suites and all that. But as long as you stick with the automatically-registered ones, it doesn't require a lot of work to use. (Then again, I haven't used CppUnitLite, so can't really compare).Not trying to "convert" anyone though, just curious because if it turns out other frameworks are easier still, I might as well use one of those instead. [grin]

Despite adding in fixture support to my own version of CppUnitLite, I've changed my mind and think that explicit fixtures like this are a red herring in C++. The old boost method of adding fixtures (mentioned in my second link, presumably before explicit fixtures were added) is IMHO better:

struct MyFixture{  MyFixture()  {    // .. setup code  }  ~MyFixture()  {    // .. tear down code  }};TEST(Foo){  MyFixture fix;  // .. test code ..}

- Setup/teardown called exactly before a test starts, even if you inherit from another fixture or have other objects as members.
- Doesn't require any explicit support from the unit test framework, making it simpler (especially useful if you're using macros to create tests since it reduces the number of creation macros you need).
- Easier to debug and step into if something goes wrong.
- Can easily add multiple fixtures with explicit initialisation ordering.

Explicit fixtures with setup()/teardown() calls make more sense in languages like Java when you don't have auto objects, but for C++ I can't see why you'd do it any other way.

##### Share on other sites
Quote:
 Original post by OrangyTangAdvantages: - Setup/teardown called exactly before a test starts, even if you inherit from another fixture or have other objects as members. - Doesn't require any explicit support from the unit test framework, making it simpler (especially useful if you're using macros to create tests since it reduces the number of creation macros you need). - Easier to debug and step into if something goes wrong. - Can easily add multiple fixtures with explicit initialisation ordering.Explicit fixtures with setup()/teardown() calls make more sense in languages like Java when you don't have auto objects, but for C++ I can't see why you'd do it any other way.

Yeah, I think it's mostly a matter of convenience. The Boost.Test docs mention the same thing, that explicit support for fixtures was added very late, because it just isn't all that important in C++.

The docs mention these issues with it here:
Quote:
 * We need to add a fixture declaration statement into each test case manually. * Objects defined in fixture are references with "." prefix. * There is no place to execute a "global" fixture, which performs "global" setup/cleanup procedures before and after testing.

#1 is a fair point. Boost.Test allows you to register a fixture for an entire test suite. Declaring local variables as you did requires you to add it to every test case manually. Not a big deal of course, but it can save a bit of work.
#2 is of course nothing more than convenience, so no big deal either way.
And #3 is another valid point, something that can't easily be done without support from the testing framework.

But yeah, fixtures aren't essential in C++, and any fixture registered with Boost.Test could just be used as a local variable instead. (Or vice versa, the class you just defined above could be registered as a test suite fixture, since Boost.Test just relies on constructor/destructor too)

##### Share on other sites
Quote:
 Original post by Spoonbender#1 is a fair point. Boost.Test allows you to register a fixture for an entire test suite. Declaring local variables as you did requires you to add it to every test case manually. Not a big deal of course, but it can save a bit of work.#2 is of course nothing more than convenience, so no big deal either way.And #3 is another valid point, something that can't easily be done without support from the testing framework.

Number 4 was the deal-breaker for me - CppUnitLite (and most C++ frameworks) use macros to define each test case. Since you can't (in a cross-platform way) overload the macros based on number of arguments you end up with a big mess of macro names, especially when you want to allow multiple fixtures to be used at once:

TEST(Foo) {}TEST_WITH_FIXTURE(Foo2, FooFixture) {}TEST_WTIH_FIXTURE_2(Foo2, FooFixture, BarFixture) {}TEST_WITH_SUITE(Foo3, SuiteName) {}TEST_WITH_FIXTURE_AND_SUITE(Foo4, FooFixture, SuiteName) {}TEST_WITH_FIXTURE_AND_SUITE_2(Foo5, FooFixture, BarFixture, SuiteName) {}// etc. etc.

Like most unit testing issues it's a trade-off between ease of use, flexibility and verbosity. I would much rather take the simpler and more flexible TEST and TEST_WITH_SUITE over the macro mess above even if it comes out a bit more verbose in the end. YMMV.

##### Share on other sites
Quote:
 Original post by OrangyTangNumber 4 was the deal-breaker for me - CppUnitLite (and most C++ frameworks) use macros to define each test case. Since you can't (in a cross-platform way) overload the macros based on number of arguments you end up with a big mess of macro names, especially when you want to allow multiple fixtures to be used at once:

Yep, I agree with that one. [grin]
Having to keep track of the different BOOST_REALLY_LONG_MACRO_NAME_THATS_SIMILAR_TO_FOUR_OTHER_MACRO_NAMES macros is a pain... [lol]

##### Share on other sites
Quote:
Original post by Spoonbender
Quote:
 Original post by OrangyTangNumber 4 was the deal-breaker for me - CppUnitLite (and most C++ frameworks) use macros to define each test case. Since you can't (in a cross-platform way) overload the macros based on number of arguments you end up with a big mess of macro names, especially when you want to allow multiple fixtures to be used at once:

Yep, I agree with that one. [grin]
Having to keep track of the different BOOST_REALLY_LONG_MACRO_NAME_THATS_SIMILAR_TO_FOUR_OTHER_MACRO_NAMES macros is a pain... [lol]

*Ahem* excuse the flagrant advertising spiel:

test-o-matic uses variadiac macros (it detects whether they're available) to provide this functionality:

TESTFIX("test name", fixture_mixin1, fixture_mixin2, fixture_mixin3){   // ...}

It's not entirely cross platform as already mentioned, but works on at least the following compilers:

* msvc8 and later
* gcc 3 and later
* borland C++ builder 2006 (aka bcc32 5.82)
* digital mars dmc (from some very early version)
* reasonably recent versions sun studio (untested though, I don't have this)

So it's not *that* un-portable.

[Edited by - the_edd on October 23, 2008 8:00:18 PM]

##### Share on other sites
Quote:
 Original post by the_eddSo it's not *that* un-portable.

Depends what your platforms are. :)

##### Share on other sites
Quote:
 Original post by SpoonbenderYou do know that manually registering your tests and all that is optional, right? They have a bunch of macros available for automatically registering tests, and even for organizing them in test suites.

I'll be damned. I cannot imagine how I missed that my first time through. Thanks.

##### Share on other sites

This topic is 3340 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.