Engine DevLog

What I thought might be a cool idea is to create a dev log all about making a game engine from start to finish, I'm not talking just a simple triangle in OpenGL.. I'm talking about everything, from building the API around the renderer to particles / phsyx / post processing and even advanced features, I one day want to implement like a cut down V-ray rendering mechanism.

I'll include everything, all the trials and tribulations. The points of utter frustration, the days where I worked so long I can't even remember my name the next day.. I'll approach it from a beginners perspective.

Why not just use a tutorial?

Well, I'm not going to release all the code because it'll one day be a commercial engine. Even though it's me on my own doing it out of work, also what tutorials generally don't give you is the headache of debugging. They don't include common issues and things you may come across, neither do they go into the actual experience of doing all of this.

What's my experience?

I've been a hobbyist / professional developer / engineer for 14 years, spanning across multiple different sectors including games / telecommunications / retail etc.

Am I a beginner?

Ummm, yes and no.. I've done many projects large and small, I've built engine frameworks (or rip-offs for learning) and worked on games as a developer and artist. But I will be starting from scratch to create a modular engine with as little interclass dependency as possible. So additions and features don't need a re-write, the whole cause of a previous failure. This is a new one for me too, so it'll be very interesting.

Can I do this?

Yes, if you put the time in.. It's hard work and in many years I have never seen a genius programmer neither have I ever seen someone build an engine with no experience of game creation before. I have seen slackers and hard workers with a thirst for knowledge..

The code and target

The point in this is to create a modern OpenGL framework, I'll introduce you to the framework / concepts and tools. What I want from my code is:

> Little in the way of bugs

> Easy for others to understand

> Maintainable easy framework which can suitably modified

> Doesn't crash

I'm not proud and I'm a human, I don't care if someone thinks my code is crap or not.. I do care about my end users.

Where to start:

Might be a nice introduction, go play a game.. Your favorite game (yes you heard me), instead of focusing on how fun it is.. Start thinking about how it's put together.. I'm talking make notes on what happens when you press a key and you move forward, what happens when a GUI menu item is clicked etc. etc.

There's a name for what happens, you move forward (Transformation), you rotate (Quaternion transform), you press a key (Event).. Think about the logic..

If I get hit, then I loose life.. Hmm you say an if statement..

Also a primer on how this all came to be might be a good idea:

http://openglbook.com/the-book/preface-what-is-opengl/

What language do I choose?

Well being honest, the language is kind of irrelevant in the whole scheme of things. But I will be choosing C++, I'll be straight up and say I don't really like C++.. A lot of the quirks infernally annoy me, on basic render tests there has been near 0 impact on performance on a modern PC between C++ and C# with garbage collection.

So why?

Because most of the source examples I have reverse engineered are in C++, even from the Nvidia developers framework, also you might as well get used to it as GLSL shaders are in C anyway.. it's easy to find a deferred rendering example in C++.. But using an OpenTK wrapper and using C# examples are scarce and it can become painful. Also for better or worse, it's what I'm used to.

Will I use libraries?

Of course I will, you'd be mad to create an input and sound library.. Even if you're in a team of 20, any shortcut, ANYTHING that can make your life easier use it.. Building an engine is hard enough and takes years, let's not add to the pain. Everything else is all us, the renderer, shader and toolset will be done by us.

What do I get out of this?

I'm starting from scratch and it's a give and take scenario, some Guru's on here might help me in certain area's.

I hope this is a good introduction and I'm looking forward to updating as I go along.

As a fellow EngineEngineer (I'm doing one but WebGL in HaXe language) here are some cents.

If you're focusing in the end-user you should try C#, as you said the perfomance will not be an issue (look at Unity3d), simple and frequent operations like sorting, delegates, get/set methods and other hi-level stuff done in 1 C# line will help a lot.

Just wrap OGL stuff in a static class and you are free to create a higher-level class library that make the calls organized and the GC is a good friend too.

In terms of class modelling I found Unity's way really awesome (Entity-Component based), you have a container (Entity) and Components are attached to it, after that, only destroying it will remove it (it is nice to avoid lost references).

I have a 'Object' like class (Resource) which contains useful stuff like unique-id, name string, ... which I register in a global Resource list, useful to simply destroy everything after a scene changes or globally search any stuff based on type or name (Reflection is another awesome feature of C#).

I organize stuff in scenes wich is basically a XML [dependecies + hierarchy] that create stuff in RAM/GPU memory and also creates the entities in 3d space with its components + reference to the loaded dependencies. When a new scene is called, destroy everything and restart the process.

About OpenGL, I suggest also following Unity's footsteps. Working with the concept of Renderer + Material, I reached a way to sort the rendering stuff based on properties of both elements and then optimizing stuff and fitting RenderTarget operations (E.g. ImageEffects) more easily in the pipeline.

Lastly, I will reveal the feature that took most of my time in the 6 months of development. The Collada Loader adapted to correctly load Bones and Animations. It was a real pain. Also the Skinning technique was hard to make it work efficiently in the GPU mode.

Nice post! Keep in touch!

**Edit**

Also create a static class to draw any kind of gizmo on top of everything (lines, wirespheres,wirecube,...) it will help a lot to debug hard stuff (skinning)

This was exactly the original purpose of my site.

It was supposed to be a logging of the trials and errors of constructing an engine, not to have a bunch of tutorials. It eventually shifted focus because there was no reason not to include tutorials and other things, but until now I haven’t really talked a lot about the “trials and tribulations” part. That is changing now that I am redoing the graphics module.

Although my philosophy on using 3rd-party libraries is different, we basically seem to have exactly the same goal with our sites, so readers will be able to see different ways of doing it all, which I would consider a good thing.

L. Spiro

Sounds great, If you don't mind can I use some of that blog as excerpts? Anyway I'll be going down the OpenGL strictly path.. For a couple of reasons, A) the mantle OpenGL derivative and B) cross compatibility..

From a quick scan, you seem to have quite a lot of cool stuff that I'm interested in.. But I'll not be including mobile support, because I don't have a clue .

This is literally a high end renderer and toolset for good looking games.

@ Eduardo

Exactly what Sean says, the engine and core concepts will be written in C++.. Then a layer of abstraction for C#. CryEngine is written in C++ but you can use CryMono for example as an end user layer.. So that's the path I'll go down.

Also appreciate the input, XML might be a cool way to go for external resources..

About doing in C++ allowing C# scripting, I found it great!

My experience making these two communicating is really shallow so I could not use this technique!

My point is that making the end-user working in C++ nowadays isn't that productive anymore.

Using XML as scene manager was my choice to avoid binary data being loaded with XMLHttpRequest (HTML5+WebGL Engine).

Other data is a mix of XML and Base64 bytes compressed using LZMA.

So a 'mesh' would be:

<mesh>

<vertex>base64_float_bytes</vertex>

...

</mesh>