I'm trying to understand unit testing. More specifically, I'm trying to understand how I would use it to improve the quality of my C++ based project. Now, I think unit testing is an amazing idea for a purely functional language. Every time I write a function I'd write a unit test over the entire input domain of that function (or some set of representative values), run the function through the test, fix cases that cause the function to return an incorrect value and move on to the next function. But how would I do this in an imperative language? At any give time (and probably more often than not) I may have a function that modifies some object's state. Usually the object is a member variable of some complex type and the function I intend to test doesn't really return a testable value: its only purpose is modification of state. How is unit testing used in this scenario (which tends to be the case vast majority of the time)?

Craftsman to the Rescue!

Hi,

Well, basically, for each test you would create a new instance of the object you are testing and then call some function that modifies its internal state, and then call some function that indicates what state the object is currently in, and test that.

For instance, suppose you are writing a Stack class. Then, you could have it look like this:



Then, you could write unit tests which test the following things:

- Push something onto the stack. Peek to make sure that we get back the same thing.
- Push something onto an empty stack. Check height to make sure it's 1.
- Peek at the stack. Make sure height didn't change.
- Push something onto the stack. Pop it back to make sure you get the same thing.
- Pop something off the stack. Check that the height is now one less.

That's sorta how it goes, only more complex for more complex objects.

There's also an article on test-driven development here:

Test Driving Expression Template Programming

Check it out. It may help you too.

Vovan

Quote:Original post by Washu
Craftsman to the Rescue!

Which articles should I look at? It seems like there are quite a few and none that answer my question [smile]

They all answer your question, it's a series, on unit testing. Where they develop and application and examine how unit testing works in a realworld situation.

The reading is very light, and easy to understand and follow.

Ahh, thank you. I didn't realize it right away. I'll post some comments/questions after I finish reading.

You are testing the class as a whole, not individual functions. Specifically you are testing that the output of the class is correct under all possible scenerios. If you can prove that then you passed the unit test. Some function could be "wrong", but if it has no impact on the output under any scenerio you still pass your unit test. Things like variables that are written, but never read are caught by the walkthrough, not the unit test. If you miss it in the walkthrough you should catch it when developing, not running, the unit test. An example is that your unit test should test every branch of every if statement, but you can't create a scenerio where a particular branch is actually taken or you can't create a scenerio that allows you to verify the value of some variable. So you have dead code or a variable that is written, but never read or perhaps never used at all.

I am sure there are probably many different ideas about the term unit testing, but here was what I decided to define it as and use it in a professional environment.

In essence, unit testing is the idea of testing your overall code base by exercising the individual components separately. I would not recommend using this in lieu of a formal testing process (black box/white box/functionality verification/usability) but rather as a mechanism to enhance the formal testing process.

If you skip into your "three weeks to alpha" point in development and start to fling builds to QA, you will humbled to find hundreds if not thousands of bugs (if you are a one man show – then the process is purely masochistic). Many of these problems will be resolved by one code fix, others are symptomatic of larger unknown issues. Think bugs in the memory management code, for example.

Unit testing, at least how we practiced it, was used to accomplish two goals: First, I wanted to involve the QA team as deeply into the project as possible. This meant that when we began the full test cycles, they would be better able to break the code as well as describing what broke. Second, developers extensively testing their own code is a waste of time IMHO. A quick check to ensure that the task is completed correctly is fine, but I need development building features. However, it is of great importance to thoroughly test all code paths - so we worked with QA and built simple command line interfaces to code modules and allowed them to hammer the code.

We also discovered other benefits - we began to track performance on individual pieces of the code and where we knew there would be bottlenecks were are to architect solutions with great ease. In addition, we also knew we could thoroughly trust large portions of the code base early on, thus eliminating them from scrutiny when other problems arise. This meant that our dev/qa meetings now would be informed discussions about where bugs may be and how as an overall team we could find and resolve the problems.

The level in which you decide to do you unit testing, I believe, is subjective. I would think it is certainly plausible to exercise code at the class/object level all the way to something like the "render engine" level. The true benefit is being able to find esoteric problems and boundary criteria early. it is much easier find and resolve a memory alignment problem early, vice when you are in the beta cycle.


Mind you, this is how we chose to integrate this process into our team. I am sure there are +/- to our technique, but honestly I think its success was more influenced by the fact that we all bought into the idea and tried to make it work vice the idea itself. We all came to realize that we created better code more quickly using this process – even though we had to spend more time building interfaces and test harnesses to accommodate the process. However, these tools were used all the way through the process and QA made frequent use of them when trying to isolate problems. The bug reports that QA would provide were always concise, direct and pointing to the exact area where the problem would exist. Overall – I would say I am a big believer in this technique.

I hope that helps -

#dth-0

Quote:Original post by xiuhcoatl
In essence, unit testing is the idea of testing your overall code base by exercising the individual components separately.

As far as I understand it, the idea behind unit testing is testing components at the lowest reasonably possible level of functional completeness (can you parse that sentence? [smile]) For functional languages this level is a function. For object oriented languages this level is believed to be a class. The problem I'm having with the whole thing is that in my mind most bugs arise from complex interaction of components and cannot be exposed by testing each class individually. If you attempt to test higher level code it becomes impractical to write unit tests because of the enormous amount of different scenarios that need to be tested.

Perhaps I can give some examples that can illustrate my point better. First of all, certain things cannot be unit tested. Code that requires user interaction is one area where unit tests fall short. All the automated tools are extremely expensive and AFAIK aren't very good. Graphical output (say of a 3D engine) cannot be united tested because I don't have nor could I acquire reference data to compare my unit tests to. Now, suppose I am writing a system like e-bay and I'd like to test some part of the API responsible for handling auctions. How would I unit test my code on that level? The process involves extremely large set of timing dependant asynchronious inputs. How would I write unit tests to emulate such an environment? I'd need to write hundreds of potential test cases. Now, if I try to test the system on a lower level (components that the auction API uses to get the job done) I won't catch most of the bugs because they arise precisely from component interaction, not from localized functionality.

So basically, how would I use unit testing to test component interaction with thousands of potential test cases (that could cause the app to take thousands of different code paths), and how would I use it in situation where it isn't possible to immediately peek at the component's state? (A single call to the scene graph modifies the render queue state which modifies the renderer state. How do I unit test that?)

you have it backwards, you don't write the function and then the test, you write the test and then the function.