• entries
    626
  • comments
    1446
  • views
    1008245

Insert Scream of Terror

Sign in to follow this  
ApochPiQ

121 views

My brain really wants to crawl out of my skull and go... someplace else.


My VS2005 task list nicely summarizes the situation:

TODO: write an entire 3D engine


This is going to be one heck of a massive undertaking, and the sheer power of it is just overwhelming. On the one hand, I'm rewriting technology that exists already, which means half the hard part is done - I know the functionality that has to be supported, I know how that functionality works, and I know what it takes to implement it. Even still, rewriting tens of thousands of lines of complex engine code is no cakewalk.

There are some deeply profound implications to making this kind of change to a very old codebase. A great example is the use of fixed-point math routines throughout the pipeline. Fixed-point was a good idea when the first foundation code was written - 12 years ago. In modern times, it's half as fast as floating point support, and requires a lot of MultiplyFixed() type crap instead of just using a * b - because, when the engine was originally written, C++ was pre-98-standard and not really a choice for use in a commercial game engine. That means no operator overloading, which means ugly code.

Rewriting the engine means moving out of fixed-point crap; but the problem is, a huge amount of code is written to assume fixed-point data. The net result is that the bulk of the game's 400,000-line codebase is affected by having floating-point data in the 3D engine core. Some of that can be mitigated by taking advantage of existing interfaces, where we can just do case-by-case conversions of data between fixed and floating point. But it's still going to force us to rewrite close to 70,000 lines of code to handle the data format change.

Another deeply pervasive problem is memory allocation. We have a reasonably decent allocator that does things like tag what section of the game is allocating memory, checks for leaks, and so on. The problem is it's written in the C style of allocation - malloc()/free() - which means that the vast bulk of the engine does a lot of manual memory management. That will all go away with proper use of STL containers, RAII, and some other tricks. The amount of dynamic allocation we do per frame is utterly incredible right now.


The work is more than worth it, though: it means that we no longer have to do 16 fixed-to-double conversions for every single transformation matrix on every single chunk of geometry in the entire game... once a frame. Maybe we can also pull out the shader-parameter code that's doing dozens of string comparisons per geometry chunk per frame.

I'm really starting to understand why the game is so amazingly CPU-bound, even with the outstanding graphics. It's a testament to the enormous talent and perserverance of the team that we've even kept the thing running this long.



Another sticky point comes up here. Since we are effectively reconstructing the entire scenegraph system and internal geometry storage/handling mechanisms, the existing D3D code will cease to work - it relies heavily on the old scenegraph design (such as it was) to know what data to push to the hardware. So we can't actually see the results of this engine rebuild until the render logic is retooled - except the programmer who knows the rendering code is already drowned in work, so that may be a while. In the meantime, we can't use the new scenegraph to get visible results without maintaining a parallel copy of the data in the old scenegraph - which utterly negates the point of doing the new one in the first place. So we have a sort of mutual-dependency thing going on, and the net result will be a few weeks of total blind work.



On the positive side, we now have an official codename for the engine: Phoenix. I made a joking reference to taking the existing engine and "burning it to the ground." From the ashes rises a new and much nicer engine... et voila, cheesy pun.

Hopefully I'll actually be able to produce something that is an improvement and not a net loss [smile]
Sign in to follow this  


5 Comments


Recommended Comments

Sounds like no small challenge [oh]

But, maybe I'm just strange, but there is something "refreshing" about starting-over every now and then. A spring-clean if you like [grin]

Good luck!

Jack

Share this comment


Link to comment
Actually we use a fair bit of middleware already, although not as much as we really could; and we will be using more in the future. Lots more.

But the 3D engine we need is utterly different from the needs of... pretty much every middleware engine out there. Any existing middleware that is generic enough to handle outer space (vs. the kind of stuff needed for Yet Another Goddamn FPS) is also too underpowered to achieve the visual quality standards that we've set. Really we're one of the few cases left where it genuinely is better to roll our own 3D engine than to try and buy one.


But that's just the 3D engine. We use third party compression schemes, video stream systems, XML parsing and validation, image loaders... and some stuff I can't mention. But there's definitely a decent bit of not-our-code in the game, as well as the content pipelines, and that amount is certainly increasing.

Share this comment


Link to comment
My eyes sort of glazed over reading your task list and i came to rocking back of forth crouched under some table. I certainly do not envy *that aspect* of your position. :p

Share this comment


Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now