Jump to content
  • Advertisement
Sign in to follow this  
daviangel

What's up with all the unit testing?

This topic is 4053 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I've been programming game since the DOS days and have hundreds of game programming books and lately all the books I've bought like "XNA Pro game programming" are starting to use unit testing. When did this become popular and why is everyone into unit testing or have to spend a chapter or so covering it? It is really that helpful that everyone needs to cover these days or is it because of ruby on rails? I say this because I also read my first ROR book not long ago and half the book was spent on unit testing and I didn't feel that I was even doing any web programming at all except testing and more testing! I guess I'm corrupted by all them Lamothe game programming books I read in the DOS days that did no error checking at all to keep the code size down and kept the code less cluttered so a newbie wouldn't get distracted by it. I guess I feel unit testing is distracting me from my main goal which is to make games. Anyone else feel this way or run into unit testing lately?

Share this post


Link to post
Share on other sites
Advertisement
More likely it's the relatively recent development and integration of unit testing frameworks which don't suck. Doing some unit testing is a great help in achieving your goal, shipping and maintaining bugfree software. That said, doing it too much stops supporting that goal. I've heard not a few cases where people went a little overzealous with it, hindering actual development for little benefit.

All things in moderation.

Share this post


Link to post
Share on other sites
Quote:
Anyone else feel this way

Absolutely not.
Quote:
or run into unit testing lately?

Absolutely.

With dynamic languages like Ruby, Unit Testing fills a gap/void left behind by the absense of static type checking.

With a static language like C++, Unit Testing fills the huge gap that comes about from the total lack of diagnostic errors for undefined behavior, a plauge that afflicts any nontrivial C++ program and and is the cause for much suffering during debugging -- not to mention mentioned every second sentance of the C++ standard as being the behavior of something or other. Unit testing brings my debugging phase down by huge gobloads.

Unit tests are also extremely straightforward to write -- usually taking a few minutes at worst to test a given component. Most of my time in writing unit tests is spent debugging the undefined behavior I managed to uncover, which I would've had to debug later anyways -- most likely in a far more confusing context.

Share this post


Link to post
Share on other sites
Quote:
Original post by MaulingMonkey
Quote:
Anyone else feel this way

Absolutely not.
Quote:
or run into unit testing lately?

Absolutely.

With dynamic languages like Ruby, Unit Testing fills a gap/void left behind by the absense of static type checking.

With a static language like C++, Unit Testing fills the huge gap that comes about from the total lack of diagnostic errors for undefined behavior, a plauge that afflicts any nontrivial C++ program and and is the cause for much suffering during debugging -- not to mention mentioned every second sentance of the C++ standard as being the behavior of something or other. Unit testing brings my debugging phase down by huge gobloads.

Unit tests are also extremely straightforward to write -- usually taking a few minutes at worst to test a given component. Most of my time in writing unit tests is spent debugging the undefined behavior I managed to uncover, which I would've had to debug later anyways -- most likely in a far more confusing context.


I've never actually tried unit testing (correction... tried it once several years ago and it seemed too difficult to implement)

But on a scale of 1 to 10, how much effort/difficulty would there be in setting up unit testing (such as boost's unit testing package) for a reasonable C+ code base (probably 150 source files... average size 70K)?

Would it involve a lot of code mongling? If not... I'd like to try it again.

Share this post


Link to post
Share on other sites
Quote:
Original post by Verg
But on a scale of 1 to 10, how much effort/difficulty would there be in setting up unit testing (such as boost's unit testing package) for a reasonable C+ code base (probably 150 source files... average size 70K)?

Would it involve a lot of code mongling? If not... I'd like to try it again.


#include <boost/test/unit_test.hpp>
#include <boost/test/included/unit_test_framework.hpp>
using boost::unit_test::test_suite;

void one_equals_one() {
BOOST_CHECK_EQUAL( 1, 1 );
BOOST_CHECK_EQUAL( 1, 2 );
}

test_suite* init_unit_test_suite( int , char *[] ) {
test_suite *test = BOOST_TEST_SUITE( "example suite" );
test->add( BOOST_TEST_CASE( one_equals_one ) );
return test;
}



Running 1 test case...
./main.cpp(7): error in "one_equals_one": check 1 == 2 failed [1 != 2]

*** 1 failure detected in test suite "example suite"
Press any key to continue . . . _


In VS2005, this should compile as-is with no extra configuration. Some of the other tester macros.

If you've got a large, thoroughly debugged codebase, the main thing unit testing makes easier will have already been dealt with for what you've already written -- namely, debugging. Hopefully, anyways. Instead of building a big up front test of everything, I'd instead introduce tests as:

1) I add new code to the repository, using unit tests to reduce the costs of debugging new code.
2) I refactor old code, using unit tests to reduce the costs of debugging when I break something.
3) I debug code, using unit tests to help check possible causes.
4) I squash bugs, setting up unit tests to flag the problem when it breaks next.

If you've got a mess of a repository that management won't let you throw out, of course, my original assumption doesn't hold, and it may well be worth unit testing everything in existance :).

Share this post


Link to post
Share on other sites
My development time is split roughly as follows:

20% Implementing unit tests.
20% Implementing new functionality.
50% Refactoring older functionality.
10% Debugging.


I've found that this distribution of time is twice as productive as my previous one:

10% Planning implementation.
40% Implementing new functionality.
20% Refactoring older functionality.
30% Debugging.


If refactoring is cheap, planning implementation on a small scale before writing it is usually inferior to writing it before planning, because written things are easier to handle and rework than things in the mind. Because of this, most of the additional implementation time in my previous approach was wasted in circumventing existing design mistakes, with refactoring being applied (quite costly) in situations where only a near-complete rewrite was possible.

However, cheap refactoring is nearly impossible without unit testing, because otherwise you run the risk of breaking the behaviour of the code in subtle ways. Unit tests help automatically debug your program on-the-fly as you alter it. The time spent writing them is saved twofold:
  • It replaces the "planning" step of usual iterations, by being a way to lay down the design of your code in a "eat your own dog food" fashion, which usually makes the design better.
  • It replaces the manual testing you would have to do every time you alter the code.

Share this post


Link to post
Share on other sites
Unit testing isn't just about being able to say "it works", it's about driving the development from your actual needs rather than what you think you need. You write the test for something first, and that tells you what your API should actually look like. Then you can go an actually fill in the implementation and the test tells you whether it works or not.

Doing it the regular way means that you tend to write the easiest/simplest implementation based on the current internal structure. This has a tendancy for the actual API to be less useful and prone to exposing implementation details that it shouldn't be.

Share this post


Link to post
Share on other sites
There is a saying: if it's not tested, it's broken.

We write unit tests for pretty much all our code for which it might be appropriate (ie. most of it) and the unit tests must all pass before your commit is allowed into the build. This reduces the number of broken builds dramatically and leads to more uptime and more productivity for everyone, including the designers and artists - and therefore makes for a better game.

Also, if I was evaluating two potential middleware solutions, one of which came with a unit test suite and the other didn't... then all else being equal I'd go with the one with unit tests. You know it works.

Share this post


Link to post
Share on other sites
Quote:
Original post by OrangyTang
Unit testing isn't just about being able to say "it works", it's about driving the development from your actual needs rather than what you think you need. You write the test for something first, and that tells you what your API should actually look like. Then you can go an actually fill in the implementation and the test tells you whether it works or not.


This is TDD (test driven design). Unit testing is one of the methods used in this aproach.

As for everything else... During the DOS era, 640k was enough for everyone. Those times are long gone. The world has changed several times over.

New methodologies have apeared during this time, some were more hyped than others, Agile perhaps as the most notorious one.

Unit testing doesn't give you results at the beginning. It gives you small to moderate results as the project advances. But once you have your working code base, unit tests become invaluable.

It may seem like an overkill at first, but once you need to fix codebase over a year old, you quickly see the advantages. Portable development is another aspect - rather than running your game on 3 different platforms, spending 15 minutes just to get to the point where you need to test a certain detail, you simply run automated tests, then 15 minutes later view the result log.

Another advantage is working in teams. You will, no matter how good you are, or how good your management is, break other people's code. Unit testing minimizes the chances of that.

As for results: Fully applied TDD to some domains allows companies to ship products with 0 defects ( as in, literally 0, or less than 10) reported by client. This isn't universally true, but unit testing does cut down bugs drastically if applied correctly.

Share this post


Link to post
Share on other sites
Since I don't think anyone else mentioned my favorite bonus of unit-testing, I thought I should mention it. Tests work as documentation, documentation that can never get out of date (if you ever been on a longer project you know how important this is). When I'm going to start using someone else's code it is a huge bonus if I can just look at its unit-tests and see exactly how the api is supposed to be use, any special cases to be aware of etc.

To make your tests optimal for this it is important not to go the lazy route and write tests like "test_division", to test the division function. Instead write small simple tests with descriptive names. The test's name should describe what behavior is tested, for example: "division_by_zero_throws" and "division_with_integers_truncates"

There are some testing frameworks that help you get in the right mindset for this, like RSpec.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

Participate in the game development conversation and more when you create an account on GameDev.net!

Sign me up!