• entries
63
56
• views
44160

## Unit Testing a renderer

I've always been interested in Unit Testing and Test Driven Development. Unfortunately, I've never had the chance to apply this to any of my professional projects. So I decided to try it out at home. Since my interests lie with graphical programming, and general software architecture, I've decided to apply Unit Testing and to a lesser degree Test Driven Development to my refactoring/rewriting/implementation of multiple renderers. Initially only on the PC platform under Windows, but later also Linux and non-PC platforms. Different renderers would be OpenGL 1.5 era, OpenGL 3.0 minus deprecated functionality, Direct3D 9, and Direct3D 10/11. A software renderer might be a nice exercise, but I'm not sure I really want to do that. For non-PC platforms I want to write a DS renderer (homebrew).

The first question I asked myself was "How does one Unit Test a renderer?" That seems fairly obvious; you generate pictures and compare them. What to compare them with? I decided I would initially generate a picture, manually verify that it is what I expect, then archive it, and from then on compare with the existing picture. This works nicely for refactoring, but the initial process isn't quite 'Testing'. However, I don't see a way around it really. Since especially the DS renderer will be low-resolution I've decided to keep these images smallish at 128x128 resolution.

To compare the images I initially wrote my own comparison routines which compared the images pixel-by-pixel. This worked well until I moved development onto a different PC. At that point the images generated by the renderer suddenly weren't matching the stored images. After looking at the image closely it turned out that in the WhiteTriangle image below the horizontal and vertical edges were each extended 1 pixel, so the edge at 45deg was one pixel to the right and one pixel up. This totally broke my image comparison. First I thought one of the images was incorrect because of driver issues, but after posting on the forums Brother Bob mentioned something about OpenGL allowing variations. After a look in the latest OpenGL spec it turns out he's correct:
Quote:
 OpenGL 3.0 Spec, Chapter 2.1The GL is designed to be run on a range of graphics platforms with varying graphics capabilities and performance. To accommodate this variety, we specify ideal behavior instead of actual behavior for certain GL operations. In cases where deviation from the ideal is allowed, we also specify the rules that an implementation must obey if it is to approximate the ideal behavior usefully. This allowed variation in GL behavior implies that two distinct GL implementations may not agree pixel for pixel when presented with the same input even when run on identical framebuffer configurations.

What to do now? Clearly I needed to compare the images differently, allowing for some variation. Since I wasn't interested in writing image comparison routines I took a look on the net. Happily I found a solution: PerceptualDiff. This GPL library does some kind of perceptual-based comparison on two images. After writing the interface between my code and PerceptualDiff, the 'errant' PC happily reports the images are identical. I'll need to keep an eye on it as I don't know how different images need to be to trigger a failed comparison, but so far it's working nicely.

The process has already uncovered a latent bug in my file loading routines which has been there for years, and numerous bugs in my own image comparison code (which is scrapped now anyway). Overall I'm quite happy with how things are going. Now the process is working, I'm going to attempt to write at least one test a day (on average). This should get the total test count up pretty quickly, and hopefully the test-coverage will go up with it.

Here are the first few images I'm testing against, so far only with a basic OpenGL 1.5 era renderer.

Default clear color (set inside each renderer to be dark gray)

Custom clear color

A simple triangle (both Projection and Model/View matrix are set to identity)

A translated triangle (Projection matrix is still identity)

A scaled triangle

A rotated triangle

Combinations of the above matrix operations are nice simple tests to add in my daily additions.

An isocahedron (a 3d mesh, mostly in preparation of adding lighting in the next tests)

## OpenGL 3.0

I recently got a laptop with a videocard capable of OpenGL 3.0 so I wanted to quickly test it. I found out that there aren't many test programs on the net, so I made one myself. It mainly tests GLSL 1.30, with geometry shaders. Nothing fancy, but if it runs, then it means OpenGL 3.0 is basically working.

Get it here.

It's based on NeHe and uses GLee for linking extensions. Source is included.

## Awesome app idea

Work on the 4e5 game is going slowly, but surely now. I've been implementing a few of the game mechanics, and it seems to work (coding-wise, not necessarily gameplay-wise).

On another note, I had an idea for an awesome app this morning.

It's an FTP tool, that allows you to drag files onto it's icon, which are then uploaded to a specific (customisable of course) ftp site. When you right click on the icon you get two extra options: download, and clear. Download would download whatever is at the ftp site to you pc (customisable path both at ftp site and local), and clear would simply clear out whatever is at the ftp. With such a simple interface, you could easily use this for basic file transfer from home to work for example.

Now someone please tell me an app like this already exists [grin]

## Thoughts

I've been quite busy lately, working on a variety of things, except what I should really be working on: 4e5 [grin]

1. Since my 4e5 game will be slightly fantasy based, and I'm going for a graphical style similar to Warcraft3, I figured I'd just write a loader for Warcraft3 models to use as temp art. So said, so started. Unfortunately, the format is barely documented. I did find one document which describes all the blocks, though not what they are for. Bar a few errors in that, I've been able to load the static base mesh, and am now in progress of loading the bones for the animation (I think). Will post screenies once I've actually added code to convert to my internal model format.

2. Abstracting renderer into DLL. I've always wanted to try this, so I decided to give it a go. My renderer was already quite self-contained because I work for multiple platforms, so it shouldn't be too much of a problem. I got the basic D3D sample off internet, and made it into a DLL. With a little help from Dave with compiling it, it seems to work. It doesn't do much yet, but it does work.

3. Something else I've been thinking a lot about lately is materials and vertex streams. Since my code is meant to work on machines which differ in power greatly, I need to get this right else it'll be a mess in client code. My current idea is to keep models and materials internal to the rendering system (which is modular) since it wouldn't usually be needed outside of that. I'm not going to worry about physics just yet [grin].

4. I've been using RakNet to add some multiplayer to my 4e5 entry. It'll make it easier to test the gamelogic and balancing while there is no AI yet. Connecting works, next step is to actually share the commands.

5. boost::any rocks! I figured I needed something to allow me to pass some arbitrary data into the DLLs for setup routines. I vaguely remembered boost had something, so I did a quick search. It turns out boost::any is exactly what I needed. Observe:

PropertyList:
#include #include #include class PropertyList{public:	void Insert( const std::string& name, boost::any value )	{		propertyMap_[name] = value;	}	void Remove( const std::string& name )	{		std::map::iterator iter = propertyMap_.find(name);		if(iter!=propertyMap_.end())			propertyMap_.erase(iter);	}	template< class T >	bool GetValue( const std::string& name, T& value )	{		std::map::const_iterator iter = propertyMap_.find(name);		if(iter!=propertyMap_.end())		{			try			{				value = boost::any_cast(iter->second);				return true;			}			catch(const boost::bad_any_cast &)			{				return false;			}		}		else			return false;	}private:	std::map propertyMap_;};

Usage:
PropertyList myList;myList.Insert("astring", std::string("mytext"));myList.Insert("anint", 5);myList.Insert("afloat", 3.5f);std::string st;int i;float f;double d;bool success = myList.GetValue("astring", st);		// returns truesuccess = myList.GetValue("afloat", d);			// returns false, and doesn't modify dsuccess = myList.GetValue("anint", i);			// returns truesuccess = myList.GetValue("adouble", d);		// returns false, and doesn't modify dmyList.Remove("astring");myList.GetValue("astring", st);				// returns false, since astring removedmyList.Remove("me");					// no-op since "me" isn't there

It allows you to store any data in the PropertyList and access it extremely easily.

## 4e5 #1

Last time around I focused on graphics very early on during the contest because I wanted to create levels for a platformer using the Q3 map format. I also spent a lot of time implemented (faulty) collision response. And so never finished, or even got started on the game.

This year, I'm going to do it the other way around. I'm making a strategy RPG similar to Final Fantasy Tactics Advance, but in a different setting of course. The game lends itself to a more iterative development, so I decided to be smart. I'm in the process of making a prototype of the game, which looks ugly, but has the basic gameplay in it.

In the current state I have a 'map' and a number of players with different stats which can take turns. The plan is to have moving and attacking done by tuesday. After that I'll start implementing the reward and level systems.

A shot of the current state below:

## Daedalus progress

Although it's been a while, I have been able to do quite a lot of things.

I've uploaded a few demos which I made about a month ago to test some of the more advanced OpenGL functions. Click the images to get the executable(s). (edit. Unfortunately, I currently check for supported extensions at init, regardless if they're used or not. I'll update the FBO and VBO samples this evening to not check for PBO support. Until then, the samples only run on nVidia cards with the correct drivers)

VBO
This is pretty much a standard implementation of VBOs in OpenGL. Not much to say about it actually.

FBO
First implementation of OpenGL Frame Buffer Objects. These allow OpenGL to render to a texture. Very useful for things like mirrors and such. In this sample, I'm rendering the scene to a texture, then using that texture to index into a 3d noise texture. The darker the color, the more noise, and thus the darker the resulting image. The effect is ok, but is a bit spoilt by the spinning of the model I think.

PBO:
OpenGL now has Pixel Buffer Objects on nVidia cards. This allows OpenGL to use a vertex buffer as a texture basically. It can be used to make a particle engine on the GPU, as I've done here. The particles update is controlled through the pixel shader (written in GLSL in my case). The two noisy texture on the left are the two textures I'm ping-ponging between for the update. rgb correspond to xyz position of the particles. The red line on the top right of the image are the particles. The shader currently updates them along the line of parabole. However, since the particles are initialised at random, changing the shader can lead to some other interesting effects.
Note: this demo only works on nVidia cards, as Ati cards don't have support for PBO yet.

Besides these visible changes, I've completed most of a new resource manager, which is threaded. Still deliberating a bit over the interface. The manager immediately gives you back a handle which you can use to access the resource. If you access a resource that hasn't been loaded yet, it's loaded ASAP. So in theory there's no need for the user to know that the resource manager is async. I'm wondering if my interface should reflect this. Any opinions on this?

Also, I've been working to load OpenGL dynamically, instead of linking to opengl32.lib. It's been relatively easy to get all the pointers, unfortunately for me, the entire thing comes crashing down if I actually remove the dependency. For some reason wglCreateContext fails. After a lot of searching I've found a small project doing this also, which does work. And doesn't do anything significantly different as far as I can see.

Lastly, I've been making a lot of progress on the Nintendo DS build of Daedalus. It can now load models and textures, and use them with an OpenGL like syntax.

## Doh, and small improvements

Doh: the 1/2 pixel differences was due to me using a rotate for my own renderer, but not on the OpenGL version. So that's fixed :D

A few small improvements have been made: a TnL cache that can cache the last few vertices. This seemed to have a small improvement in my test cases, but nothing really big. This is more or less what I expected, since I'm using flatshaded models, which obviously don't repeat stuff. It did increase the speed a bit if I used models with non-flatshaded normals (but these looked like shit).

I'll be adding code to calculate normals on the fly, and see if that gains me anything with respect to the original version.

Still need to look at the matrix stuff, so I'm slowly learning some ASM.

During that I did a quick test to see if there was any difference in ASM between functions written my fixed point class (with operator overloads and whatnot) and my fixed point typedef + helper functions. None!, so I'll now happily convert everything to use the class, since it obviously makes the code much clearer.

I'm starting to switch to a new testcase which draws 4 cubes instead of one. The FPS dropped to 1/4 of what it was, which is mostly to be expected, but was still a little bit disappointing, as I had hoped it would drop slightly less.

## Almost there ...

Spent some more time with my renderer yesterday evening, and there are now no more cracks. A case of using floor where I should have been using ceil [oh].

However, all is still not well. For some reason I have a 1 or 2 pixel offset when comparing my output to OpenGL's output. I don't think this is down to the accuracy loss when going to fixed point math, so I'll be taking a look at my matrix routines next.

## Current state ? Improving

I've not done a lot of programming lately, so there's not much to tell in that regard.

- Use GBA specific faster memcpy functions
- Do two pixels at a time when possible to take advantage of the 32bits buswidth of the GBA (1 pixel is 16bit)
- Moved code into IWRAM (which is the fastest location possible on the GBA, but it is fairly limited)

I've now got the same image as in the previous post running at twice the speed (38fps). A simple cube of the same size is now running at 70-80fps.

Unfortunately, while testing, I found out my renderer isn't quite correct. Some pixels were being plotted twice. Since I broke the renderer in windows when switching from floating point to fixed point. So this weekend I spend some time cleaning up the fixed point code to the point that I can now run it in windows again. To make sure I was getting the results right, I compare my output with OpenGL output. This gave me some initial problems because I couldn't get OpenGL to disable anti-aliasing. After some help from Sages I changed settings in my Display Settings. Although this didn't seem to help at the time, it seemed to work the next time I worked on this (after a restart). So if anyone has problems like that: a restart might help you out.

The next step will obviously be to fix my renderer (sigh), which is what I'm doing at the moment.

A number of optimizations I'm still planning to do are:
- Matrix functions. These seem awfully slow at the moment. Unfortunately, I'll need to dig into ASM for this, something I'm not looking forward to.
- Switch to indexed colors. The copy from backbuffer to framebuffer will go twice as fast (or maybe won't be necessary, since the 256color screens natively support double buffering). I'm currently planning a game that should look fine with 256 colors. Unfortunately, doing something like this will probably break the general usabilty of my renderer, so I'll have to do some thinking on how to implement this within the current code.
- After I've made the switch to 256color I'll need to change the renderer to be able to plot 4 pixels at once (to use the full 32bits in the bus). This should give some increase in speed in my testcases, although I'm a bit worried that my game will not really benefit from this as the triangles I'll be rasterizing will only be a few pixels each.
- T&L cache. The current testcases are flatshaded, and will therefor benefit not one iota from this, as each vertex needs to be different anyway to accomodate the separate normal. However, if I'm willing to calculate the normal for each triangle inside the renderer, I can drop this, and plot a cube using only 8 points, instead of 24 (using quads). This should be a nice boost, but again I'm holding off on this one because it will break the generality of the renderer.

As the situation is right now, I think it's almost time to start the game code as well, so I can get a good feel for how fast is will run in actual conditions as opposed to testcases.

## GBA Renderer

Someone was asking about the GBA Software Renderer, so here I present it to you in all it's glory:

It's nothing much yet, but it supports clipping and backface culling. The thing that I need to look at next is the matrix math, it seems awfully slow.

For those wanting to see it in action, download a copy of the .gba file plus the emulator from here

## Progress

After the last post I've done quite a lot. The silhouette module is now pretty much complete. It won't be easy to get it to go faster except perhaps by implementing it with a vertex shader (which I would like to try at some point).

I've also started homebrew GameBoy Advance development, and I've been surprised by how easy it is to get things up and running on the actual device. It's quite cool to say the least. Unfortunately, the GBA has no 3D hardware, and no 3D API, so after searching the net a bit I decided to learn and program my own 3D software rastizer. Progress for this has been ok. I've got Gouroud shading working correctly. However, my fill algorithm isn't quite correct, so I'm taking out a lot of time to fix that. Hopefully I'll make pictures of this soon.

Lastly, there was a GameDev gathering in London last week. It was quite fun to meet more of the people from the forums. I did take a few pictures (click for larger):

C J W and csarridge

phantom, Sandman and Superpig

phantom, Sandman(without camera) and Superpig

polly, Monder and MarkR

Clockwise starting at bottom left: Boruki, C J W, Superpig, Monder, polly, MarkR, Sandman, Phantom and csarridge with *PIES* in the middle.

Thanks to all of you for a great time, and I hope to do it again soon.

## Silhouette edges 2

After a week of little coding, I've more or less finished the silhouette module. The detection of which edges to use is complete now, and I've changed the algorithm generating the borders so the borders are more even. It also takes into account the distance to the model, so the border is always the same thickness regardless of distance. However, this last change seems to break sometimes. Need to investigate and fix that before I call this one finished.

## Still alive

Well, it's been a while with no updates.

I've been playing Every Extend a lot, and inspired by that have created a silhouette detection module. It works most of the time, there's just a border case I need to fix.

Work on the 4E4 entry has been going slow, so we probably won't make it. But none of us is really concerned though, so it sounds like we may keep working on it after the entry.

## Programming

Now that I've found a bit more time to program I'm glad to say I'm getting quite some stuff done. Unfortunately not on the 4e4 entry though :D.

I've been working on a system identification library for the past week. It's going ok, but it's difficult to test since I only have a single system myself.

Also I've been able to add drag-and-drop onto the Daedalus engine. So now I can drop model files into a viewer and stuff like that!

## It is done!

I've sent my thesispaper and report off to my supervisors yesterday evening! Now I only need to do a presentation, and then my studying days are officially over! Woot.

## Untitled

No fix for the quaternions yet, although I've got a thread running here.

Got back the first version of my thesis paper, and while my supervisor agreed with the things I said, I still need to add and tweak a lot of stuff. So I'll be extremely busy on that in the evenings :(

## Gah, BSP collision detection

So for the past few evenings I've been working on fixing my BSP collision detection. I found out it was too simple to just take the bottom of the current leaf as the floor :)

So I've been testing all kinds of stuff, and it just wouldn't get it right. I've been going through a simple level by hand and it seemd to go to the wrong leaf at the end. Then yesterday I finally found out I had to get leaf number -(leaf+1) instead of -leaf.

Then this morning I checked the docs to see if that was in there as well, and it was :(. That'll teach me to read quickly! But at least now it's working :D Next stop is fixing the quaternion interpolation in the Doom3 models.

## Working animation

And after another few hours of coding, the animations are also working correctly:

## Hard at work

Well, this is my second week on the new job and it's been pretty fun so far. Bar the occasional bug of course.

For some reason I've been inspired to work on my own stuff in the evenings as well, even though I still need to finish my thesis (can't wait to get that done, another month to go).

I'm seriously contemplating working/finishing an entry for the 4e4 contest. It'll be a basically 2d platformer but using a 3d engine. I'm using Quake3 levels for the levels, and Doom3 models for the player models at least, and maybe for static objects as well.

I've got the Quake3 map loader working image. I've created that test map in Qeradiant, and I've already been able to add some custom entities (the boxes). Detecting what the height is under a player position is a breeze with the bsp structure, and one of the floors is colored slightly blue here to give a visual indication of that. Also the bsp planes are shown.

The Doom3 mesh loader is working correctly as well: image, and I think the animation loader is also correct (image), but I haven't gotten animations to work yet. But then I've only spent two evenings on the doom3 code, so I'll get there.

Happy coding to all you, and until next time.

## GLSL and GUI

I've decided to comment the source files heavily as that will keep everything in a single place. I should however also create some diagrams of how different parts of Daedalus interact.

I was having some motivational problems the past weekend, so I decided to dust off my old GLSL code, and refactor it into the new version of Daedalus. I'm happy to say it went pretty well. It didn't take too much time, although conforming to the resource system of Daedalus isn't as easy as I thought it would be, so that might need another refactor some time in the future.

I'm currently moving my GUI code back into Daedalus, but I'm not happy with the results. The GUI is too dependent on other things. Luckily, I've been working with someone to make a simple GUI library for general use, so I'll be reworking the GUI soon anyway. The largest problem I foresee is linking the GUI with some kind of font system.

Just read about the 4E4 contest, and I'd really love to participate. We'll have to see how that goes.

The Daedalus-source is getting quite a lot of downloads, more than I expected in any case. Unfortunately, I haven't had any feedback yet though.

The source to my implementation of the paper 'A Practical Analytic Model for Daylight' has also been downloaded relatively often already. But again, no comments :(

## Daedalus grows

Having a severe cold the past weekend, I decided not to go outside and enjoy the fine weather but stay inside and code.

I fixed a few things in the windowing code, and most of the functions seem to work both on Windows and Linux. I also readded my ResourceManager to Daedalus, and proceeded to test multiple contexts with this manager. Naturally they didn't share textures at first, but I'd been expecting that. After adding a simple call to wglShareLists/changing the glxCreateContext both windows can access the same texture.

I've decided to try and push Daedalus more for other people to use it. As such I will need to find a license for it, and add *a lot* of comments. I tried out Doxygen and NaturalDocs last week, but I wasn't too happy with them. I like NaturalDocs, but it didn't generate documentation for the classes automatically. I had to comment each and every variable. Doxygen was okay, but I don't like the output. So either I try and find out if I can change the output format of Doxygen, or I heavily comment the source files. I'm leaning towards the second one at the moment.

For the license I'm going to use something liberal like the zlib license. I would like to get feedback from people who use it, and 'fix' things, so I'll go and do a little bit of searching on that.

Just for those willing to try it out, you can download Daedalus from here. The zip-file include two 'test' programs. Makefiles are provided for Linux, and VCExpress Beta 2 solutions for Windows. If you do try it out, please let me know what you think of it.

## Still alive

It's been awhile, but now I've finally found the time, and got some news.

The work on the thesis is going pretty well. I should be testing with the first pilots by the end of the week. I'm looking forward to it :D

I've also had a chance to work on Daedalus a little bit. I've refactored parts of the windowing code, and I've recently dug into the windows messaging queue a bit more. A fullscreen app is almost working correctly (Alt+Tab, minimizing, etc). Once this works on windows, I'm moving over to Linux again to fix any issues fullscreen might have on that platform as well. I've added some functions to the window class to allow the user to generate these messages as well (so you can minimize the fullscreen app from within the app itself), and added virtual functions so users can take action if these things happen.

After this is working, mouse interaction will be added in again. Shouldn't be much of a problem I think.

I do wish I had some way of developing/testing this stuff on a Mac as well, but alas. It'd be much easier to port to Mac now, before I add a lot of other stuff (even though that other stuff should be platform indepenent).

On a sidenote, War of the Roses just wentover 100 downloads in the Showcase, woohoo. Makes me wonder how many people have downloaded it off my site.

My thesis work is coming along nicely, although I have been held up by some very unsightly bugs.

Daedalus is shaping up very nicely. A code snippet for the interested:
#include "System/Window/window.h"#include class MyWindow : public System::PlatformWindow{public:  MyWindow() : right(100)  {	MyWindow() : right(100)	{		mKeymap.RegisterMapping(				DKEY_ESC,				Functor0(this, &PlatformWindow::Close),				Input::Keyboard::KEY_DOWN);		mKeymap.RegisterMapping(				DKEY_BACKSPACE,				Functor0(this, &MyWindow::Left),				Input::Keyboard::KEY_PRESSED);		mKeymap.RegisterMapping(				DKEY_ENTER,				Functor0(this, &MyWindow::Right),				Input::Keyboard::KEY_PRESSED);		mKeymap.RegisterMapping(				'G',				Functor0(this, &MyWindow::Color),				Input::Keyboard::KEY_PRESSED);	}	float r,g,b;	float right;	void Color( void )	{		g += 0.001;		Redraw();	}	void Left( void )	{		right -= 0.1;		Redraw();	}	void Right( void )	{		right += 0.1;		Redraw();	}	void Draw( void )	{		glViewport(0,0,mWidth,mHeight);		glClearColor(0.f, 0.f, 0.f, 0.f);		glClear( GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT );		glMatrixMode( GL_PROJECTION );		glLoadIdentity();		glOrtho(0, mWidth, 0, mHeight, -10, 10);		glMatrixMode( GL_MODELVIEW );		glLoadIdentity();		glColor3f(r,g,b);		glRectf(0,0, right, 100);	}};int Init(){	MyWindow *p = new MyWindow;	p->Create();	p->r = 1.f;	p->g = 0.f;	p->b = 0.f;	p->MoveToFront();	return 0;}

This basically does exactly what you'd expect of it. The Init function is called from within Daedalus. It creates a window of default size and colour depth, sets the color and moves it to the front (thereby giving it focus).
The window registers couple of input functions, and implements a drawing function.

This code works on Windows and on Linux without changing a single line. Only a recompile is necessary.

## War of the Roses v2

Well, it's taken a while but it's finally up. Download the new version of War of the Roses from here, and please let me know what you think of the AI difficulties.

## War of the Roses AI

I've checked the AI for War of the Roses today, and it's looking good. It needs some minor tweaks, but I think it's pretty nifty. Thanks Risujin!

I'll be posting the new version here with the next entry, which I expect to be sometime this week.

Removing the GLFW code has been quite painless so far, and I'm learning a good deal about X11 while creating the Linux port. Still need to get started on the Mac port though.