Resource Manager Design

Started by
17 comments, last by haegarr 9 years, 3 months ago

Hello,

few days ago i started writing my own Resource Manager. It's not my first attemp to do this so i'm trying to design it as good as possible, but i run to couple of issues.

So i've got Resource<T> class that takes as template parameter some data.

For example if i want to pack Shader inside it , it's done like this:


typedef Resource<ShaderData> Shader; //Somewhere in code
Shader SomeKindOfShader = ResourceManager->load<Shader>("Basic"); //This returns existing shader or loads new
SomeKingOfShader->bind(); //-> is overloaded to return ShaderData

But the problem is in ResourceManager, to make it most flexible i decided to make ResourceLoaders as Depedency Injection objects.

Let's say:


class ResourceManager
{
    public:
        void addLoader(ResourceLoader* NewLoader); //Adds it to map
    
    private:
        std::map<typeid, ResourceLoader*> Loaders;
}

class ResourceLoader
{
    public:
         virtual any load(void* Data) = 0; //Loads given from memory data
}

class TextureLoader : public ResourceLoader
{
     public:
         any load(void* Data)  //Here is problem, loader would like to know also about Width, Height etc.
         {
               //Loads and returns data
         }
}

The problem is that i have no idea how to workaround problem with additionall parameters for given loaders.

Texture loader here is just example, there could be Shader loader who would need also compilation parameters for uber shader etc.

I've come with ideas of using va_list, initializer_list and even variadic_templates but they're not really beautiful solutions.

Here is my question: How to design(or redisign) current way of managing those loaders with custom number of parameters?

Thanks for any help :)

Advertisement

So i've got Resource class that takes as template parameter some data.

Each type of resource should have a manager. Trying to abstracting everything it's a waste of production time; you can even got something, but at the end you'll end up with a more complicated and unmantainable architecture.

In your case, you can have a templated map class that delete all pointers when it's time to shutdown the the application. Example from my old project:


#ifndef MAP_H_
#define MAP_H_

#include <map>

template< typename _tKey, typename _tVal >
class CMap {
public :
	inline ~CMap() { Reset(); }

	inline void Reset() {
		for (std::map<_tKey, _tVal>::iterator I = m_mValMap.begin();
			I != m_mValMap.end();
			++I) {
			delete I->second;
		}
		m_mValMap.clear();
	}

	inline _tVal Get(const _tKey& _kKey) const {
		std::map<_tKey, _tVal>::const_iterator I = m_mValMap.find(_kKey);
		return I != m_mValMap.end() ? I->second : NULL;
	}

	inline void Add(const _tKey& _kKey, _tVal _vVal) {
		if (_vVal) { m_mValMap[_kKey] = _vVal; }
	}

	inline void Remove( const _tKey& _kKey ) {
		_tVal* ptVal = Get(_kKey);
		if ( ptVal ) { 
			delete ptVal;
			m_tMap.erase(_kKey);
		}
	}
protected :
	std::map<_tKey, _tVal> m_mValMap;
};

#endif

...and on each resource manager, you can have a map of each base resource class.


But the problem is in ResourceManager, to make it most flexible i decided to make ResourceLoaders as Depedency Injection objects.

You can explicitely tell the correspondent resource loader to request other resources as a dependency. Example:

CTextureManager uses a CImageManager.

Pass the image manager to request a image and create a texture from that.


I've come with ideas of using va_list, initializer_list and even variadic_templates but they're not really beautiful solutions.
Here is my question: How to design(or redisign) current way of managing those loaders with custom number of parameters?

Would be better separating these responsabilities for each type of resource. Each resource can be inherited from a base class. Example:

A texture can be a two-dimensional texture or a render target, but both resources need to be managed differently even if they're directly related. So, you should create a texture manager and a render target manager, even if your texture it's inherited from a render target. A texture manager requests images from the image loader, so you'll need to explicity define that, passing a image class pointer to the texture manager (if you're using a instance based architeture) or requesting directly the image from the image manger.

How about just using a variadic paramter pack? In the simpliest way, you can just use a map<std::string>, where you store a variant-class, like boost::variant.


class ResourceLoader
{
    public:
         virtual any load(void* Data, const std::map<std::string, boost::variant>& Params) = 0; //Loads given from memory data
}

class TextureLoader : public ResourceLoader
{
     public:
         any load(void* Data, const std::map<std::string, boost::variant>& Params)  //Here is problem, loader would like to know also about Width, Height etc.
         {
               boost::variant& Size = Params["Size"];
               //Loads and returns data
         }
}

You could typedef the map or wrap it in a class, but the concept is the same. You'd also have to lookup how boost::variant (or the variant class of your choice) works.

How about adding a void* to a struct with specific parameters to your load() method?

class ResourceLoader
{
public:
    virtual any load(void* Data, void* Params) = 0; //Loads given from memory data
}

struct TextureParams
{
    int width;
    int height;
    //and whatever extra info you want
};

class TextureLoader : public ResourceLoader
{
public:
    any load(void* Data, void* Params)  //Here is problem, loader would like to know also about Width, Height etc.
    {
        TextureParams* textureParams = (TextureParams*)Params;
        textureParams->width;
        textureParams->height;
        //...
    }
}

devstropo.blogspot.com - Random stuff about my gamedev hobby


Each type of resource should have a manager. Trying to abstracting everything it's a waste of production time; you can even got something, but at the end you'll end up with a more complicated and unmantainable architecture.

I strongly agree. Storing inhomogeneous data typesafely and without unnecessary typecasting is difficult in C++. The most natural way is to make one manager per type of resource. This also works in other similar problems, such as in entity component schemes (one manager per component).

You may want to check my recent thread http://www.gamedev.net/topic/663222-resource-manager-for-open-world-game/ . I'm still in the process of making a generic resource manager framework myself.

Depending on your requirements, you may want to consider the use of handles to mediate resource usage.


How about adding a void* to a struct with specific parameters to your load() method?

I discourage this approach. It's the "C way" and is not typesafe.

I made a generic resource manager, and I find that it's working quite well.

I have an XML file where you define the resources, and you can add custom properties to your resource class.


<Container name="sprites">
    <Container name="player">
        <Resource class="Sprite" name="run" path="run.png" frames="8"/>
    </Container>
</Container>

You need to override the initialise() and load() functions:


void Sprite::initialiseParameterTemplate() {
	addParameter("frames", Parameter::Type::Integer);
	setParameter("frames", 1);
	setParameter("class", "Sprite");
}

void Sprite::load() {
	m_texture = new Texture();
	m_texture->setParameter("path", path());
	m_texture->load();
}

But after you've got that done, it's really just plug and play.


addResourceClass("Sprite", [](){ return new Sprite(); });
loadResourceHeader("resources.xml");
loadResources();
Sprite* sprite = resource<Sprite>("sprites/player/run");
int frames = sprite->frames(); // 8 - it returns "sprite->getParameter("frames").Integer" 

Is there anything with this design that could really make things break later on?

I hear so much bad about going this way, but I feel it's just perfect.

I discourage this approach. It's the "C way" and is not typesafe.

A void* works. I also don't like it that much, but without going full template mode, it's the most simple solution to the problem the OP presented with minimal code impact.

IMO, a better way would be to stop trying to homogenize/generalize resource management until you have written non-generic managers for every resource type you're using, and have stopped adding additional code to them. Only then can you safely say "ok, these have a lot in common, maybe i can generalize this part a bit". But by then, you have working managers, so what's the point biggrin.png

I hear so much bad about going this way, but I feel it's just perfect.

Try adding sounds, scripts, entire spritesheets, texture atlases, meshes, fonts, etc. Write dummy code just to prove to yourself it doesn't break.

devstropo.blogspot.com - Random stuff about my gamedev hobby

I hear so much bad about going this way, but I feel it's just perfect.

Try adding sounds, scripts, entire spritesheets, texture atlases, meshes, fonts, etc. Write dummy code just to prove to yourself it doesn't break.

Yeah, I've got several classes in my library I've already tested with: sounds, music, UI style files, textures, sprites, fonts, meshes, shaders..

And I also have made a game level as a resource in a game, and it worked as expected.

I'm just curious why people say it's bad.

One of the criticisms of "manager" classes is that "manager" is an overly broad verb, that doesn't actually say what the class does.
Usually people end up using such vague verbs, because the class is in violation of the SRP and is responsible for too many things, such as interacting with the OS's filesystem, caching loaded files, referencing counting files, deserialization/serialization, file-parsing, etc, etc...

Personally I like to break apart this uber-manager into it's individual parts like this:

BlobLoader -- just loads blobs of bytes from disk/wherever. Does not know what the data is. The load function takes a callback to call on completion. Can be an interface/virtual if you have different kinds of file-systems, e.g. if during development you use a network filesystem instead of a local archive one, etc...

BaseFactory<T> -- implements common functionality for factories that convert blobs of bits into usable objects (parsing/deserialization/etc). NOT an interface (no virtual) - used via private/implementation inheritance.

BlahFactory : public BaseFactory<Blah> -- The specific factory for each type. If the load function requries width/height, add it. There's no interface inheritance, so BlahFactory can have a different Load signature to FooFactory.

AssetCache -- handles the situation where you want to avoid loading a resource twice. Keeps track of which files have been loaded before.

e.g.
BlobLoader loader;
AssetCache cache;
FooFactory fact_foo(loader, cache);
BlahFactory fact_blah(loader, cache);
Foo* foo = fact_foo.Load("foo");
Blah* blah = fact_blah.Load(13, 37);
If you then wanted to make an uber-XML parser, which can load any kind of object from a file, you'd make a new class for that, which makes use of all your other classes. You'll probably need an adaptor to tell the XML factory how to use the other's though, e.g.
MegaXmlFactory fact_xml( loader, cache );
Foo* FooFromXml( FooFactory* subFactory, const XmlNode& node ) { return subFactory->Load(node["name"].AsString()); }
Blah* BlahFromXml( BlahFactory* subFactory, const XmlNode& node ) { return subFactory->Load(node["x"].AsInt(), node["y"].AsInt()); }
fact_xml.AddSubFactory( fact_foo, &FooFromXml );
fact_xml.AddSubFactory( fact_blah, &BlahFromXml );
This keeps your code reusable and maintainable. Each component does one small task well, without knowing too much about other components. The XML factory doesn't even know the type of the sub-factories that it's dealing with, and there's not even a sub-factory interface! Each sub-factory can be implemented in a completely different way, without being forced to be part of some inheritance chain. The loader can be tasked with just talking to the OS file-system without being over-complicated with any other tasks, such as what to do with the data afterwards - it doesn't know who/what is on the other end of it's OnLoaded callbacks. Likewise the AssetCache knows nothing about the loaders or the factories, it's just used by the factories to avoid unnecessary use of the loader.

I know this is controversial, but to be honest, that just makes it a lot more complicated than it needs to be.

It is probably because I'm still a novice, but I just feel adding tons of small classes do nothing more than make everything messy.

Also, all my resource manager actually does is:

* parse the XML file

* call the "load()" function of a resource that is going to be loaded

* get resource by name from a vector of ResourceContainer which has a vector of Resource.

Sorry for being stubborn, but I need some serious convincing to see the faults of this.

This topic is closed to new replies.

Advertisement