Sign in to follow this  

Assert, exceptions, and unit testing

This topic is 3861 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

This is a pretty convoluted problem, so bear with me. I have 3 projects in my MSVC 2005 solution. One is a GUI library I'm developing, another is that library's unit tests, and a third is a sample application I'm building using the GUI library. I'm building the GUI library to throw exceptions when things go bad (say, the user specifies a negative value for a widget's position). All's great, and my unit tests are checking that those cases are throwing exceptions. However, I'm finding that the debugging value of assertions is better than exceptions (which unwind your entire call stack). Ideally I'd like my GUI library to throw an assertion before the exception, so I can debug those issues in the IDE if possible before my application handles the exception and erases the call stack information. Now the tricky part. The unit testing needs to be automated, so I don't want it to perform any assertions when I'm deliberately checking pathological cases to ensure they're throwing exceptions. The GUI library is in it's own library project. Both the final application and the unit tests link against it. The library therefore needs to behave differently depending on whos linking to it. I honestly have no idea how to proceed, or even if I'm barking up the wrong tree here. All I want is for my project to shutdown gracefully, and with a good error message, when I distribute it, but to throw assertions so I can debug it while I'm developing it. But this assertion behavior needs to be supressed during unit testing.

Share this post


Link to post
Share on other sites
Firstly I think that exceptions should not leave a library boundary. The library should be the one to handle those cases - it alone should know what to do - and use return values or error codes to reflect problems back to calling code.

The second part is a bit confusing to me. I'm assuming you want to test if assertions were fired while unit testing.

In that case, introduce an AssertHandler. This class or function pointer handles all assert handling and can be replaced by other implementations. So by default your handler would check the condition and report if it turns out to be false, while when you're unit testing, a different handler will be used that records the false condition but does not stop the application. The unit test can then question the handler to see if an assert has indeed fired.

hth,
CipherCraft

Share this post


Link to post
Share on other sites
It is perfectly valid to have exceptions that aren't handled in the library it self. There's no need to do some ugly C-ish return codes - if you want to have exceptions - it's totally your call!

As for asserts, there's an excellent article at Power Of 2 Games site. You should get a few ideas about your custom assert macro. Also, read the comments - there are some good suggestions how the code in the article could be changed to be more cross-platform.

Share this post


Link to post
Share on other sites
Quote:
However, I'm finding that the debugging value of assertions is better than exceptions (which unwind your entire call stack).


You do know that you can tell Visual Studio to break when an exception is thrown (ie. before the stack unwind) regardless of whether it's caught, right? The default behaviour is to only break on unhandled exceptions. To change this go into the 'Debug' menu, 'Exceptions...', and configure it as you like. If your sole reason to use the assertions is to break before the stack unwind, you don't need them.

Share this post


Link to post
Share on other sites
Quote:
Original post by joanusdmentia
Quote:
However, I'm finding that the debugging value of assertions is better than exceptions (which unwind your entire call stack).


You do know that you can tell Visual Studio to break when an exception is thrown (ie. before the stack unwind) regardless of whether it's caught, right? The default behaviour is to only break on unhandled exceptions. To change this go into the 'Debug' menu, 'Exceptions...', and configure it as you like. If your sole reason to use the assertions is to break before the stack unwind, you don't need them.


I didn't know that! Thanks a bunch. Most of what I know in MSVS is from either figuring it out on my own or reading articles on the internet. Good to learn something new!

Share this post


Link to post
Share on other sites
One thing to keep in mind is the origin of unit testing.

It gained most of its momentum during the Java's 1.3/1.4 era. In that system, only exceptions can be thrown, and even asserts are exceptions.

As such, the usual unit testing design doesn't allow for anything else.

C++, due to its static compilation and pre-processor is much less feasible to test.

Another important aspect is the behaviour of classes. Unit tests should strive to automatically test all possible code-paths - there should be nothing left for manual examination.

If you want to get full benefit from automated tests, then make sure tests do really test everything, including triggering of asserts, and if possible or not done already replace asserts, at least in test builds, with an exception.

Unit testing allows you to test a case where exception is thrown.

Another important point is that unit testing should ensure that classes during run-time do not encounter anything that isn't unit tested. Or - your debugging value of asserts should be 0. If you need to manually run through the code and check for various conditions, your tests are faulty, your classes improperly tested and your code sub-par (not a critique, just consequence of unit testing).

Ideally, during run-time, you should never encounter a situation that isn't described in tests already - if you do, write a regression test for that very situations, reproducing the problem. Then, fix the problem and make sure the test passes.

As for different linkage - your library should have an interface which is consistent - if it doesn't, that's likely a design issue. Even if linked incorrectly, it shouldn't crash or do anything unusual, just fail at pre-determined points. No matter how it's built.

Rigorous and consistent unit testing has (proven on real projects) the ability to make code run and ship on first or any build. Since each part works and is tested 100%, and since all compositions of these parts are tested, the sum of all this parts will work 100% as well.

But, as said, always keep in mind that Agile methodologies evolved from Java, which is a much much much different beast than C++.

Share this post


Link to post
Share on other sites
Quote:
Original post by Antheus
C++, due to its static compilation and pre-processor is much less feasible to test.


Err, I take issue with that statement. The problem with C++ is that response to errors is typically "undefined behavior" instead of "do something I can easily detect in my unit tests". This has nothing to do with either point listed.

Another "problem" is that the compiler refuses to generate a program without more skeleton code in place for a C++ program than a Java one. However, if you consider compilation of your unit test a stage of your unit test, this actually turns out to be a net benifit -- it's the equivilant of an automatic unit test that your compiler writes for you (by way of not compiling broken code). Even if you don't buy this logic, such really only applies to TDD -- Test DRIVEN Development -- and isn't applicable to unit testing as a whole.

In fact, much of the ease of unit testing in C++ derives specifically from macros, allowing the likes of BOOST_CHECK( ... ) to tell you the specific expression that failed, the source file, and the line number, thanks to #, __FILE__, and __LINE__ respectively -- all pre-processor things. Without the preprocessor, a unit test will tell me when something breaks, but it'll require a lot more manual legwork in the unit test for me to be able to figure out what -- be it stepping through in a debugger or manually managing checkpoints throughout the unit test.

Numsigl: You can break-on-throw even in GDB (`catch throw` to break on the throwing of any exception, for example, IIRC).

Share this post


Link to post
Share on other sites
For my projects I derive all my exceptions from a single base exception class that captures the stack and allows richer reporting than what(). It also provides a handy place to stick a breakpoint. This infrastructure doesn't work for exceptions out of my control such as std::bad_alloc but those usually aren't as interesting anyway.

Share this post


Link to post
Share on other sites

This topic is 3861 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this