So here's the deal. I'd like to write a graphics engine from scratch. I'm not exactly interested in the gameplay aspect of it, but only in the experience I'll gain from writing something like that. I've done OpenGL before, written things such as terrain generators, hierarchical animation and various other crap without using anything but simple OpenGL, but it's nothing I'd be proud to show. I've also used Ogre for quite a while, and I like it to a certain extent, but using it is really no good for someone trying to get experience in the low level details. I'd like to write something that would follow healthy principles of engineering and that I could work on over time to expand it, to include shaders that I'd write myself, scripting, a console and other goodies that will make it a serious project. I don't really have any problems with writing anything complex and even if it is hardcore, that's good because I'll get to learn something interesting :) My real issue is what I want to write it in a proper way, not hack together some code and realize months later that the initial design totally sucked. It's impossible to come up with a good and flexible design for a thing such as a 3d engine when you have never done it before. I've been looking around for good resources for how to face this big task, hoping to find textbooks that would pertty much guide me in this task, but not exactly write it for me :P I found this book, even though i don't know if it's any decent: http://www.amazon.com/Game-Engine-Design-Second-Interactive/dp/0122290631/ref=pd_bbs_sr_2/102-0830581-3467301?ie=UTF8&s=books&qid=1178635351&sr=8-2 I would really appreciate if you guys could give me a few tips for where to start and where I could find resources for this. Things such as online guides and books would be fantastic. Thanks for your time, sorry for the long read :) Psicho.

Awesome tips, thx :)

So would it still make sense in your opinion to go through that book to see what approach the author is taking, or is his way somehow not too modern or just good in general?

Quote:
I'm not exactly interested in the gameplay aspect of it

This is like saying you want to make a car, but aren't really keen on the drivability of it.

Quote:
My real issue is what I want to write it in a proper way, not hack together some code and realize months later that the initial design totally sucked.

Your design will suck. Live with it. You cannot get it right the first time.

Quote:
It's impossible to come up with a good and flexible design for a thing such as a 3d engine when you have never done it before.

Exactly. This directly contradicts what you just said though -- if you know its not possible to get it right the first time, why worry so much about it? That's what refactoring is for.

Quote:
I would really appreciate if you guys could give me a few tips for where to start and where I could find resources for this. Things such as online guides and books would be fantastic.

Eberly's book is an interesting read, but by no means definitive. There is no one way to build an engine properly; in fact, in the absence of requirements, you can't build a useful engine at all.

The best path towards developing an engine is to develop some games, and apply iterative refinement to the framework underlying those games. They don't need to be big games, they just need to be something playable. To sit down and try to design an engine from scratch in isolation is folly -- doubly so if you've never built any sort of functional game (or aspect of a game covered by what your engine is attempting to address, if it's something like a graphics engine). Now, you do seem to have done that already, so you're good on that point.

Without something to make use of of the engine, you can't really tell if in practice it will be as generalized, functional, abstract, useful, whatever you want. You'll just have some academic pondering and that might very well fall completely apart when you actually try to use it. You will find those most of the major middleware engines began life not as isolated engines but frameworks underlying games that were refactored into generality.

The best way to build an engine is thus to refactor it out from underneath of a number of (probably smaller) games -- or perhaps tech demos, if you're trying for a smaller scope (graphics, physics, etc) engine versus a full-on game engine solution.

I recently wrote up a similar reply to a similar question which might contain some additional information. Let me see if I can recover it from my history before I repeat myself.

EDIT:
I can't find it, perhaps I was dreaming. However, here are two threads where I expound upon some other points (the issue of building-from-games and the issue of being afraid to get the design wrong and being afraid of refactoring):
On refactoring.
On fabricating game engines from whole cloth.

The major reasons just building the engine in isolation doesn't tend to work out is that you have no metrics for measuring any goals: you don't know how usable the engine actually is if nothing is using it; you don't know how reusable the engine actually is until you reuse it. You don't know how generalized the design is until you try to apply it to a variety of practical situations. Concrete games provide you a means of defining small pockets of requirements as well as stable testing environments. You build one game with an eye towards keeping your graphics engine distinct from any game logic -- you build simple games so your development time is not disproportionally spent on the game logic versus the engine itself. Once you've done that, you look at your results and split the game logic away (but don't discard it, it's still a useful test), then try and build a different game, reusing as much of your functionality as you can and possibility expanding on it.

This approach is much more agile, much more education, and results in more interesting products as well as a proven stable, functional, useful framework, which eventually grows into an engine. It forces you to learn from your mistakes rather than trying to pretend like mistakes cannot be made, which only harms you.

In closing, I'll note that you could probably get by without building games for a graphics engine -- you could build yourself some tech demos, instead. Small applications demonstrating specific techniques, et cetera. I don't, however, recommend this, because it doesn't force you to look at your design in the large, only in the small; using a game will help keep you tuned towards the resuability of the framework and the abstraction away from the domain-specific logic of the game -- it will help teach you about decoupling your subsystems, essentially. The domain-specific logic of technical demos tend to be the same logic as that of the underlying framework (graphics, in this case), which can result in bleeding through of dependencies, assumptions and implementation details if you're not extra careful, and you won't necessarily notice this bleedthrough until you try to finally build a useful product with it, at which point it may be too late to staunch without major refactoring.

Also (yeah yeah, this is really the end, I promise), I'd note that even if you don't intend for this to ever be production-ready or available to the public, the experience you gain doing things this way will still more practical and applicable and useful than going the "build in isolation" route.

[Edited by - jpetrie on May 8, 2007 12:53:17 PM]

Good points there. When I say no gameplay I mean that I'm not interested in making a game, but more of a tech demo. I don't mind having the basic FPS functionality in it, such as say, being able to run around the environment or display a character model (maybe with animation, but that's still in a very far future). What I don't need is an actual complete game product.

Also, as I said, I agree that it's impossible to get it right the first time, but it does not hurt to start on a good path (note: I'm not saying on the "right" path :P )

Okay. I was editing while you posted, so just so you don't miss it, this is the section I added (which mentions the tech demo approach and a few disadvantages):

Quote:
In closing, I'll note that you could probably get by without building games for a graphics engine -- you could build yourself some tech demos, instead. Small applications demonstrating specific techniques, et cetera. I don't, however, recommend this, because it doesn't force you to look at your design in the large, only in the small; using a game will help keep you tuned towards the resuability of the framework and the abstraction away from the domain-specific logic of the game -- it will help teach you about decoupling your subsystems, essentially. The domain-specific logic of technical demos tend to be the same logic as that of the underlying framework (graphics, in this case), which can result in bleeding through of dependencies, assumptions and implementation details if you're not extra careful, and you won't necessarily notice this bleedthrough until you try to finally build a useful product with it, at which point it may be too late to staunch without major refactoring.

Also (yeah yeah, this is really the end, I promise), I'd note that even if you don't intend for this to ever be production-ready or available to the public, the experience you gain doing things this way will still more practical and applicable and useful than going the "build in isolation" route.

I do tend to obsessively edit my posts, heh. Anyways, good luck.

Yeah I agree with everything you say. I'd like to avoid mixing the engine with the actual game mechanics.

Hear! hear! jpetrie speaks the truth!

I've been working on a graphics engine for a long while, and I'll second much of what was said here.

This isn't intended to be an insult to you, but believe me, your first engine WILL suck :P . This is because it's very complicated and a learning process. You'll think you'll have a well planned design, but then you'll learn something new or your style will change, and your plan will then mean nothing. What ravyne2001 said about experience coming through failure is very true.

This has been particularly true for me, as I have no formal graphics programming training, so while I consider myself to be a competent programmer, I learn new things every day (though I suspect that's true for experts as well). And a good example of style changing: I recently cured myself of singletonitis, and that itself required a rewrite.

But if you're persistent, it's very gratifying. My advice is just to keep your code modular so it can be easily reused. Then for each new version, you're not so much rewriting as refactoring. And each new version will be better than the last, as you'll be building on the experience and skills you've gained.

And there's nothing at all wrong with writing an graphics engine, while not being interested in writing a full game. I'm like that too; I prefer working at a lower level. The most I do is write very simple games and tech demos for debugging.