Context

After spending several years developing mostly 2D/3D small applications in different languages on different platforms, I decided to create ex-nihilo, an object-oriented game engine with 3D graphics. My main reasons for creating my own engine were:

• I already had existing code samples (graphic display, skeleton based model, sprite, …)
• When comparing with existing engines at that time, I thought they were lacking simplicity compared to the kind of programs I wanted to write and did not offer direct support for sprites, fonts and/or user-defined GUI.

What Went Right

1 - Focus

I wanted a cross-platform game engine to realize 2D/3D turn based strategy games or simple RPG games like Final Fantasy 7. As such the preliminary features were:

• OpenGL for cross-platform graphics at least v1.2.
• Object-Oriented Design
• Scripting: simple to change on-the-fly combat resolution, to code spell/weapon effects and game events
• Import/Export a few formats: the Milkshape 3D modeller text and binary format + my own internal format for sprites and models, TGA, BMP and light length-encoded bitmaps.
• Simple physics (basic collision detection of Axis Aligned Boxes and spheres; basic navmesh handling).
• Hardware vertex transform and lighting (T&L)
• Basic Texturing
• Skeletal Animation
• Special Effects: Billboarding, Particle Systems, Simple Terrain Rendering (Hex based and square based),
• Fonts and GUI support
• Sound system (mod player).
• Embedded logging system.
• Basic Multithreading availability

2 - Planning

In my day-time job, I am used to writing software/process specifications and planning schedules. This made the process more natural to do as I knew how. Moreover, I knew what time I had available to me and planned in consequence. Even though my documents were less detailed than what I do at work (hey, that’s a hobby for me), I had defined goals to reach and found out it would require one year’s time to complete the project. I knew from the beginning it was no trivial task and was prepared for it.

Here is the simple planning I did:

• Design the architecture: 2 weeks
• Realize the core framework (interface for creating a blank Window + initialising OpenGL + basic helper functions and logging + basic multithreading interface): 3 weeks.
• Realize the basic core Renderer (frustum + renderer + math interface + Color and Light interfaces + texture manager): 6 weeks.
• Create the font engine (refactor bitmap and rendered font code, include default fonts, create font file format): 4 weeks.
• Create the basic Scenegraph (Scenegraph and basic volume generator): 4 weeks.
• Create the Mesh Model Node (refactor existing code and file loading, process animated and static models + model manager): 5 weeks
• Create the Sprite Model Node (refactor existing code and file loading, create billboarding code, process animated and static sprites + sprite manager): 3 weeks
• Create the Particle engine Node (create code, create interface): 3 weeks
• Create basic terrain rendering (refactor terrain generation code with hex and square cells, create file format): 2 weeks.
• Create the GUI interface (refactor existing code for buttons, dialog, text, progress bar, mouse widgets, create file formats for skinning and GUI loading): 5 weeks
• Create the scripting interface (refactor existing code for scripting): 2 weeks.
• Create the sound system (reuse DUMB and adapt with OpenAL): 10 weeks.

3 - Careful design

Before committing to development I took time to polish my engine architecture. Contrary to small scale projects, this one needed to be carefully thought. It took me about 3 weeks to define a satisfying UML diagram of the engine. It did pay in the long run as each new feature integration neither broke nor frankly stressed the design. So no mid-project code overhaul or architecture redesign was ever needed. This also helped refactoring previous code as I knew the interfaces and how it should interact with the engine. This was another proof of a sound design adapted to my needs.

4 - Previous experience

I was able to capitalize on all my previous experience. I had already written simple expert systems, a forth interpreter, 2D graphics renderers, 3D mesh viewers including skeletal animation, basic multithreaded resource loaders… all simple applications in different areas of the game engine I wanted to develop. This gave a great boost to the realization of this game engine and improved my refactoring skills. This confidence is also reflected from the beginning in the planning where I knew with great confidence it was not uncharted areas. Good for the morale to know before hand that tasks won’t be that difficult to do.

5 - Iterative development

To me, what was of utmost importance was that each new feature and optimisation added did not add bugs to the engine or degrade performance. So I wrote a few samples using the engine (and even editors) and each time I modified the code, I compiled the engine against all examples to check that they perform as they were designed to. All examples were also improved each time to include the latest added features: they doubled as engine tests. I also ensured the engine and the examples worked the same way on all three different computers I use:

• an IBM ThinkCentre 8429 with no added graphics card using Windows 2000.
• a home-made Pentium 4 2Ghz with an old GeForce graphics card using Windows XP Professional edition.
• a laptop AMD processor with an internal GeForce 780X graphics card using Windows XP Home edition.

6 - Engine documentation

As this was a really big scale project, I wanted to easily maintain code documentation for ease of reference in later developments using this engine. I learnt to use Doxygen and never went back. It really was simple to maintain documentation (instead of writing full descriptive documents under Word which were a pain to maintain). This also improved the reuse of the engine since I was able to develop samples two years after the beginnning of the engine development.

7 - Keeping track

I also manually kept a log of any changes I made to the engine in a simple text file. I also manually logged any bug encountered in another text file. Bug correction had higher priority than feature development. This log helped me to keep track of what I was doing and where I stopped (invaluable for a hobbyist programmer with little development time available) but also gave a moral boost when considering everything that was done and all steps ensuring the stability of the engine.

8 - Logging for debugging

When developing a multithreaded application, the only solution to trace a code or to time a code is through logging. I had explored the use of gprof for debugging purposes. Even though it is mandatory for single thread applications, I saw two drawbacks for multithread applications:

• often the debugger does not stop on the spot where the error occurred. Moreover the spot where it stops changes over some application execution.
• a few times the error was not stable when the debugger was on: the program could run normally without any bugs on one execution and crash on another.

What Went Wrong