Ressources management in a graphic engine

Started by
8 comments, last by James Trotter 18 years, 11 months ago
Hi, Since I started working in real-time 3D engines, I always wondered how do professional 3D engines (such as unreal engine, and others) manage their ressources (textures, shaders, buffers, etc.) ... My problem is that I'm the only one working on the graphic engine, and I don't have much time to think about it. And a big problem I encountered was the textures management. When I have 2 meshes sharing the same lightmap, if they are exported as 2 seperated meshes, the lightmap will be loaded 2 times which in not really optimal -__- I thought about creating a global texture management system that will at first only detect if the texture is allready loaded into the engine to avoir multiple instance of 1 texture. But the problem of redundant SetTexture calls still occurs ... Is there some way of organize an engine to sort the entities to draw by textures, or render states, or whatever ? If someone here as such a system implemented, can they share a little bit of their knowledge ? I'm really curious to know how do graphic engines manage ressource to optimize rendering ^^ .:: Paic Citron ::.
Advertisement
For the texture management part (or any resource, not just textures) I use the method described in http://nesnausk.org/nearaz/texts/d3dresources.html - it makes handling lost/reset devices in Direct3D much nicer. Actually that's probably solving a different problem than you have, but you might find it interesting nonetheless.

For the sorting to avoid redundant settexture calls I sort first by effect and then by effect parameters. An effect being a bunch of renderstates/shaders and effect parameters being textures, shader constants, etc. You could just sort the pointers to the effect objects (the pointer effectively being an effect id) and sort the void* pointers to structures containing the effect parameters using something like a radix sort. This is probably the simplest and is basically what I use. There's still redundancy in there in some cases - eg, two meshes have same effect and effect parameters except for the texture and since the texture is different all the effect parameters are considered different.

You can get quit complicated in this area - there have been a few threads in the past discussing quite general and complicated ways of doing this sort of thing but it's a bit too involved and would take too much time to implement for me.
For the texture handling in the my last project, I implemented a handle-based resource manager, based loosely on the "A Generic Handle-Based Resource Manager" article in Game Programming Gems.

All requests for textures go through the TextureManager class. When the texture manager recieves a request for a texture resource, it hashes up the filename (using the CRC32 algorithm), and checks every texture it owns for a matching hash value. (Hashing up the filename might be premature optimization, I just thought it would be a better idea than doing a string compare for every texture)

If a texture was found with a matching hash value, then that means the texture was already loaded, and the TextureManager returns a handle to it. Otherwise, the resource is loaded, and a handle to the new resource is returned.

The binaries and source of my last project are available here, if you want to have a look at the implementation. It's not hugely impressive, it's just a demo of boost::python being embedded into a 2D application, but hopefully someone will find the code useful.

Binaries
Source Code

The relevant files are ResourceManager.h, TextureManager.h, TextureManager.cpp, Handle.h, and Resource.h

With regards to sorting your render calls, to minimise state changes, I'm no expert on this, but after reading "Real time 3D terrain engines, using C++ and DirectX 9", I've decided to go with a RenderQueue for my next project.

Basically, instead of having your objects communicate with the Renderer directly, they submit themselves to a RenderQueue, which sorts them by their various states, such that the most expensive state changes happen as little as possible.
That's some really nicely written code you've got there Oxy.
OK, thx for the replies, i'll have a look at the website, and the source code. But from what you said, I suppose there's no "miracle solution" and that I'll probably have a lot of "tweaking" to do to have a working (and efficient) manager ^^

.:: Paic Citron ::.
Oxyacetylene > I did a little search on the article in game programming gems ... but there are 5 volumes, I only have the 2 latest ones ^^
And didn't find in which one this article is (even looking on amazon) Can you tell me which volume this article is in ?

Thx in advance ^^
Quote:Original post by paic
Oxyacetylene > I did a little search on the article in game programming gems ... but there are 5 volumes, I only have the 2 latest ones ^^
And didn't find in which one this article is (even looking on amazon) Can you tell me which volume this article is in ?

Thx in advance ^^
Unfortunately, it is in the first Game Programming Gems book. [sad]

Fortunately, it's also online here. [smile]
hey, thx a lot, seems like I didn't search the web enough ^^
Quote:Original post by WillC
That's some really nicely written code you've got there Oxy.

Cheers, I think it's not too bad considering 90% of it was written on the train to and from work. I'd stopped programming for quite a while, and it was the boredom of the commute that got me to start again.

Quote:Original post by paic
Oxyacetylene > I did a little search on the article in game programming gems ... but there are 5 volumes, I only have the 2 latest ones ^^

I've only got the first three myself, need to get the other two when I get some more cash!
You might want to look at how it is done in ogre, as that's a fairly recognised graphics rendering engine.

This topic is closed to new replies.

Advertisement