Resource Management

Started by
21 comments, last by Satharis 9 years, 1 month ago

So I want to code a resource management system for my (OpenGL) rendering engine.

I have many different kind of resources: images, opengl textures, vbo's, ebo's, vao's, rbo's, shaders, shader programs. (did I miss something?)

What are the coding options for building this kind of manager? I tried looking at other rendering engines' codes but they're hard to understand.

The obvious answer for resources should be something like the following code, while other resources (like shaders and images) will have their own Manager class, and not a general one.


//abstract class
class Object //I called this Object but could also be named Resource
{
public:
	Object() : _id(-1) {};

	virtual void create() = 0;
	virtual void bind() = 0;//doesn't make sense for resources like "image"
	virtual void free() = 0;

	int id() const { return _id; }

protected:
	int _id;
};

class VBO : public Object
{
public:
	VBO() {}
	void create() {};
	void bind() {};
	void free() {};
};

class Manager
{
public:
	Manager() {}
	void add(Object *obj) {
		_map.insert( std::pair<int, Object*>(obj->id(), obj) );
	}
	void free(Object *obj) {
		_map.erase(obj->id());
		obj->free();
	}

	size_t size() const { return _map.size(); }

private:
	std::map<int, Object*> _map;
	Object *_obj;
};

Is this a good approach? Are there any better approaches for resource management systems?

Advertisement

I use this approach (which make things while obeing the SRP):

For each type of resource I create a manager. The shader manager, texture manager, render target manager (to manage render target switch and shared render targets).

Vertex and index buffers they don't need a manager so I just have a model manager since gets into a model class...in the easiest way the managers only avoid to load the resource more than once; if you need to work with some kind of related funcionality there is no problem as long it is related to the resource.

A lot of people here handle it in different way; not trying to abstract everything into a single abstract class resource manager it is a good thing also. Resources manage itselves their life-times. Destruction is inside the object destructor and construction can be a function or if you use RAII for resources do it in the way you think is the best for you; I personally handle loading into functions and destruction into a function called Destroy/Reset that gets called on the object's destructor.

"Manager" is a term that is so nonspecific as to lead to all manner of functionality being lumped together in one class. Before you start designing a resource system, it's important to decide what it is actually going to do. Is it going to load resources? Is it going to be a cache for resources? Is it going to handle automatically disposing of resources?

What you have sketched out above literally does none of these things, which makes it, to my mind, not terribly useful...

Tristam MacDonald. Ex-BigTech Software Engineer. Future farmer. [https://trist.am]

It does load resources to a std::map and it is a cache for them as well. What are you seeing that I'm not?

If you try to abstract a manager you will get back to a specialized class no matter what you do. Resource management needs to be fast as well any other module in the engine, and abstracting you're forced to add complexity in your scheme at some point. A lot of engines out there has some kind of asset manager, but the class is still specialized; each resource does not extend an asset.

What do you need? Do you need to avoid loading multiple resources? Do you want to manage allocated memory? Want the user to easily interact with a resource?

Your implementation it is not wrong; each resource has its class; you only need to define what actually you want with them.

Once the resource it is created and stored somewhere there is always a upper class that manages its life-time being high-level or not.

I would recomend searching for the topic Game Engine Architecture because there arw topics that clarifies your questions in a more advanced way;all you will hear here is problably discussed somewhere; you can take this shortcut only if you want because information is always valuable here no matter if is permuted.

It does load resources to a std::map and it is a cache for them as well. What are you seeing that I'm not?

It's not loading anything, because you are requiring the caller to provide an already loaded object to insert into the map, and it's not caching anything, because it provides no interface to lookup items which are already in the map.

Tristam MacDonald. Ex-BigTech Software Engineer. Future farmer. [https://trist.am]

I have many different kind of resources: images, opengl textures, vbo's, ebo's, vao's, rbo's, shaders, shader programs. (did I miss something?)

I'd say yes.

A couple of the things you mentioned require no active management at all. Shaders and shader programs, I very much doubt you'll be loading them on the fly or ever unloading them while a game is running, and the same can be said about RBOs, FBOs and texture attachments for FBOs, these most of the time occupy a very specific role in the pipeline (say g buffer, post prossess, light accumulation buffer, etc), so passing them down to a catch all generic resource manager seems a bit of overkill to me, you'd be architecturing for situations that wont happen. At most you'll want a few functions so you can reallocate FBO/RBO/texture attachment if the resolution is changed, they're ad hoc though, no need to create a whole system to handle them. My stance on these things is always: Code for what you will need, not for what you *might* need.

Now the resources that you do want to manage dynamically are vertex buffers and textures. Those will get loaded and unloaded on the fly.

As to how to design that resource manager, I don't have much to add. I haven't implemented a fully working resource management scheme. So far the thing I have clear is that file loading should be abstracted. So while the "resource manager" provides common APIs to request for resources, the loaders themselves are created separately and registered into the manager. If the manager lacks a loader to load an specific resource, it will let you know (aka, NPE as soon as you use the result biggrin.png ). Also that caching schemes should be abstracted too, if you want a resource to be cached, you wrap the loader into a cache, and pass it to the resource manager, the manager wont know its using a cache (because it doesn't needs to).

"I AM ZE EMPRAH OPENGL 3.3 THE CORE, I DEMAND FROM THEE ZE SHADERZ AND MATRIXEZ"

My journals: dustArtemis ECS framework and Making a Terrain Generator


If you try to abstract a manager you will get back to a specialized class no matter what you do. Resource management needs to be fast as well any other module in the engine, and abstracting you're forced to add complexity in your scheme at some point. A lot of engines out there has some kind of asset manager, but the class is still specialized; each resource does not extend an asset.

I nearly started reading it wrong the first time, but this is important. You really do not want to have a base resource type that is required to be inherited from. This means there's tight coupling between resources and manager, and also that you cannot use your manager, then, with any existing class/struct or with any primitive type without first writing a wrapper class for it which inherits the base type. This leads to a lot of unnecessary boilerplate -- better would be for resources and their manager to be loosely coupled, then you can readily place any existing type into a properly-specialized manager, or even primitive types. There's no reason your manager should not readily accept, say, std::string, and there's no reason it should be precluded from accepting, say, int or bool types or C-style file handles (which, IIRC, are just integer identifiers), either. Irlan is right, also, about specialization at some point -- Its my experience also that you want separate managers for separate kinds of things (textures, meshes), rather than one uber-manager that holds all resources. You can (and IMO, should) however allow for a manager to manage an abstract type -- eg. one texture manager instead of texture2Dmanger+texture3Dmanager, etc.

Also, many people reach to singleton pattern for such managers, but I think this is wrong-headed. Do not allow your design to assume that only one manager of a single resource type (abstract or concrete) exists -- If you design such that many can co-exist, you can choose to use only one and get all the same traits of a system which only allows for one, but you cannot choose to create many if your design enforces only one.

Also, everything people have said about the specificity of "Manager" is correct, and I do not think you've clearly defined what your goals are -- Usually, a resource manager implies either a common-point-of-access, resource deduplication, on-demand resource retirement/restoration (for e.g. streaming), pooled allocation, or some combination of these things. It is my experience that each of these properties is best treated as a separate responsibility, and so, is encapsulated into its own class, and higher-level functionality achieved by orchestrating these classes together, either enforced through a higher-level class, or by policy.

throw table_exception("(? ???)? ? ???");

Ravyne, yeah that was my point.

The OP may not want to ask why singletons are bad because thats a old long history that was started since Dracula was born. Just avoid as much as possible and if you use it go with another type of design such static functions to handle the life-time of the objects; always trying to get a instance based architecture is a good practice :-).

Also, many people reach to singleton pattern for such managers, but I think this is wrong-headed. Do not allow your design to assume that only one manager of a single resource type (abstract or concrete) exists -- If you design such that many can co-exist, you can choose to use only one and get all the same traits of a system which only allows for one, but you cannot choose to create many if your design enforces only one.

So do you claim that singleton patterns are bad habit? I've seen them used many times while viewing engines' source codes.

This topic is closed to new replies.

Advertisement