Hello : )

I'm trying to improve my code and started to implement unit-testing. There are some parts that are still a bit confusing to me, especially mocks/stubs/...

Testing code that requires external libraries, e.g. a wrapper for my Lua-binding. How to test wrapper-classes? Should the Lua-binding be replaced by a mock? But then, what to test when the wrapper is depending on the functionality of the underlying API and is pretty much useless without it. Is testing a class together with a dependency an integration-test already?

One of my classes requires other classes to register for its use (event-bus). Now, how to handle this? Shall mocks register?

Another thing is a level-loader. It surely requires a dependency as a file-system-logic and my scripting-language to process through given level-file upon construction. My level-loader has a Load()-function that takes an empty level-field and populates it. Now, I want to improve the testability of this code. But as it requires other components upon construction, should I just inject real components or mocks? Since I want it to work and it will operate on the file-system, it seems weird to inject a mock that will only simulate the file-system.

This raises quite some confusion for me, but I'm confident it will go away after a bit more testing : ) And especially improve my code.

I hope this is somewhat understandable and would be really thankful if someone could help me out. Thanks for your time!

I think we covered this in the last thread. The ideal is that you only test your own code. If your wrapper just wraps a Lua function, you just need to test that the values get passed through correctly. You don't care whether the function does the right thing. The most sensible test would be to create a fake function that you wrap, which can check that it received exactly the data that was expected.

One of my classes requires other classes to register for its use (event-bus). Now, how to handle this? Shall mocks register?

Of course, that's what mocks are for. You write just enough code to test that the event bus is doing whatever event buses do. You don't use your other application classes for the test. Unless it's impractical not to, in which case you do. Don't worry about it.

Since I want it to work and it will operate on the file-system, it seems weird to inject a mock that will only simulate the file-system.

Why is it weird? You're testing the loader, not the file system.

But, again, if it's easier for you to use the real file system - perhaps because that's the easiest way to produce decent test data - use it. Is there a Secret Testing Policeman that you're worried is going to arrest you if you don't write your tests the way they want you to?

If you can use mocks, use them. If not, don't.

Mocks are ephemeral objects with functions just enough to cover your test cases. It is not the real objects but you trick your code into thinking that it could be the real ones. One important property of the mock object is that it should be configurable in how it interacts with the code you are testing.

If you are testing a level load module that accepts a filename, then you could create a mock that only loads/saves the file data from/into memory. Although, if I were you, I would use the OS temp directory for this.

edit: If other classes are part of the same package/namespace, it is okay to use the real class as part of your unit test. Suppose LevelLoader and Texture are part of the same package, and LevelLoader, in addition to the level file, requires a Texture array to load the level. I would actually instantiate real Texture objects and pass them onto the LevelLoader.

In my experience, it is quite common for developers to build integration tests or system tests and call them unit tests.

A unit test hits the smallest possible unit, and verifies small, descrete things. If the test crosses any system boundaries it is not a unit test. If it touches a disk or network or database it is not a unit test. If it takes more time than is measured in nanoseconds it is not a unit test.

I've worked with amazing unit test systems before that perform hundreds of thousands of unit tests for a large library of code, and it completed in a matter of seconds, perhaps ten seconds or so. I've also worked with terrible "unit tests" systems that required many minutes to run. They were not unit tests.

And as ChaosEngine mentioned, unit tests ensure permanence, and also serve as an example of using the small piece. They do not validate correctness because tests can have wrong values, but done correctly they will light up the build with errors when behavior changes. That's a reason unit tests should generally be limited to long-living libraries and not to gameplay code, gameplay code is often in constant flux, but library code for core algorithms will be around forever and better not change without notice.

Thanks a lot everyone. So a unit-test should really test only that one class even its use is void without other dependencies, is this right?

Mocks do not nothing but enable my unit to even work. If my Level-loader uses some Lua-binding-command to load a file to its current "loaded file storage" (or returns it to whatever), the Lua-binding-mock will just use and provide made up information. There will be no actual loading attempts of real intense file, as that would take too long but it will still pass "something", so the whole thing works.

A file_content load_file(std::string file_path) method within the level-loader would just be tested, if it actually does return something?

If I want to test the actual loading, I would have to unit-test the actual Lua-loading-component who deals with that, is this assumption right? Unless the "load from a file system" itself is not my code.

So, I really should start to use interfaces (or in C++ "base classes") for all my dependencies so I can create mocks from them? At the moment, I rarely did this, as there will be no case where I would want to e.g. switch out a Lua-component for Ruby-component. But it seems required in order to generate mock-classes. While I see the use, it seems to generate quite some clutter, hah.

But, again, if it's easier for you to use the real file system - perhaps because that's the easiest way to produce decent test data - use it. Is there a Secret Testing Policeman that you're worried is going to arrest you if you don't write your tests the way they want you to?

Haha, this made me grin (thanks for that :)). I think it is just an unhealthy "read it somewhere on some popular programmer-forum and it had quite some upvotes" and people talked about that using mocks can be a sign for code smell. But there is probably a difference between a lot and some.

So a unit-test should really test only that one class even its use is void without other dependencies, is this right?

Often they test even less than that.

You've got several questions there. There is a wonderful book on unit test patterns that the authors generously maintain as a web site as well. I'd recommend reading through at least the first part of the book, the sections listed as "The Narratives" in that link, since it covers all those questions better than forum posts would.

edit: If other classes are part of the same package/namespace, it is okay to use the real class as part of your unit test. Suppose LevelLoader and Texture are part of the same package, and LevelLoader, in addition to the level file, requires a Texture array to load the level. I would actually instantiate real Texture objects and pass them onto the LevelLoader.

I would disagree. If you do that you are no longer unit testing your LevelLoader class, you are integration testing your level loading system.

The value in unit-testing is two-fold. First, to use code to specify the expected behaviour of a unit. But the longer term benefit is to alert you to breaking changes. If you use your Texture in the LevelLoader and then introduce a bug in the Texture later, your LevelLoader tests will probably break too. Which makes it harder to track down the problem.

Actually I do want to know how far the breakage spreads. I do want to know that this bug also breaks the LevelLoader. If I had mocked the Texture class for the LevelLoader, I wouldn't know. If I added breaking specs to the Texture class, the tests for LevelLoader would still pass as it's still using mocked texture objects, then I would have to remember to update all those mocks. And as you said that if Texture class is being used in many other places, then it becomes an onerous task just to maintain your tests. Outdated tests can hide bugs too.

This is where I think the big grey area is. I am not disagreeing with you, but sometimes unit testing can be overdone. Unit tests everywhere, mocks everywhere, that as project becomes large, maintaining these tests becomes a project of its own. Although it depends per project and case, my rule of thumb is usually the less mocks the better. Mocks are usually necessary when interacting with outside resources or packages (e.g HTTP requests), but if these tests are part of the package that I am testing, then I wouldn't mind mixing package-level integration testing with unit tests.

So a unit-test should really test only that one class even its use is void without other dependencies, is this right?

Yes, that's what makes it a 'unit' test. It only tests that unit.

A file_content load_file(std::string file_path) method within the level-loader would just be tested, if it actually does return something?

Unit tests are written with knowledge of the code that it is testing. We've not seen your load_file function so we can't tell you what it should test. But you should examine the content of that function, look for ways that it could potentially go wrong (e.g. mishandling the file_path), and write tests that try to catch those.

If I want to test the actual loading, I would have to unit-test the actual Lua-loading-component who deals with that, is this assumption right?

Unit tests aren't there to test functionality, they're there to test that code in a unit is correct. Whether the resulting functionality is correct is a higher level question, usually reserved for integration/validation/acceptance tests, depending on how thorough you want to be and what terminology you want to use. But yes, you will presumably want a test of some sort for that component.

So, I really should start to use interfaces (or in C++ "base classes") for all my dependencies so I can create mocks from them? [..] While I see the use, it seems to generate quite some clutter, hah.

Yes, often changing your code to make it perfectly testable in isolation makes it into worse code, or at least messier. Advocates of test-driven design don't agree with me on this. People who were brought up with enterprise Java tend to believe everything should be constructed via dependency injection and inversion-of-control containers which basically mandate everything being built using many interfaces anyway, which in turn makes their code easy to test. Awful to write, difficult to understand, overly verbose, under-encapsulated, but easy to test. It's a good solution when you have to make a large team of mediocre programmers effective.

My opinion is that you should do this refactoring where it is easy or necessary to do so. I mentioned this in the previous thread.