Jump to content
  • Advertisement
Sign in to follow this  
i_luv_cplusplus

Resource management woes

This topic is 3403 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello. I'm writing a 3D game engine. Unfortunately, no matter what I do the resource management seems rigid and unnecessarily complicated. Currently my resource manager is a STL map with key being string and value being abstract Resource class. Now, classes such as Texture, Shader, Material, Geometry inherit from Resource, and are casted into Resource* before being inserted into this STL map. When reading the value back I just cast it to the type I need. Should I keep it this way? Or maybe I should create a generic ResourceManager, from which TextureManager, MaterialManager etc will inherit? Also, who is responsible for loading the resources? The resources themselves, the resource manager or some inbetween "Loader" class? I would greatly appreciate some advice. Thanks.

Share this post


Link to post
Share on other sites
Advertisement
I'm just going to ask, why do you cast them to Resource* before and back to their original form after storage? Wouldn't polymorphism allow you to simply store any class that inherits from Resource* as a Resource* in an STL container with no casting necessary?

Share this post


Link to post
Share on other sites
I use a class template parameterized by resource type for my resource caches, which eliminates the need for down-casting.

How are you storing the resources? Smart pointer? Raw pointer? From what I've seen on these forums at least, a map associating integers or strings with dynamically allocated resources managed via smart pointer is a pretty common way to approach the resource management problem.

Share this post


Link to post
Share on other sites
As I said, I am using std::map<string, Resource*>, Resource being the base class all resources derive from.

Templates? Do you have a small example of them? I can't think of a real use of templates for resource manager.

Share this post


Link to post
Share on other sites
Quote:
As I said, I am using std::map<string, Resource*>, Resource being the base class all resources derive from.
Well, it's not unreasonable to ask whether you're using smart pointers. Your original post suggests that you aren't, but I just wanted to make sure.

Since you are in fact using raw pointers, a good first step would be to switch to using smart pointers. This will make resource lifetime management easier, and will help prevent common errors such as dereferencing of invalid pointers.
Quote:
Templates? Do you have a small example of them? I can't think of a real use of templates for resource manager.
This will be off the top of my head (not compiled or tested):

template < class T >
ResourceCache
{
public:
void Add(const std::string& name, T* resource) {
resources[name] = resources_ptr_t(resource);
}
resource_ptr_t Get(const std::string& name) const {
return resources[name];
}

private:

typedef boost::shared_ptr<T> resource_ptr_t;
typedef std::map<std::string, resource_ptr_t> resources_t;

resources_t resources;
};

// Example of use:

ResourceCache<Texture> textureCache;
textureCache.Add("some_texture", new Texture("path/to/texture"));
// ...
boost::shared_ptr<Texture> texture = textureCache.Get("some_texture");
texture->Bind();

That's a very simple example of course, with no error checking and with minimal functionality. It does however demonstrate the technique I'm talking about: by parameterizing the cache class by resource type, you eliminate the need for down-casting when requesting a resource from the cache.

Share this post


Link to post
Share on other sites
Quote:
Original post by jyk
Quote:
As I said, I am using std::map<string, Resource*>, Resource being the base class all resources derive from.
Well, it's not unreasonable to ask whether you're using smart pointers. Your original post suggests that you aren't, but I just wanted to make sure.

Since you are in fact using raw pointers, a good first step would be to switch to using smart pointers. This will make resource lifetime management easier, and will help prevent common errors such as dereferencing of invalid pointers.
Quote:
Templates? Do you have a small example of them? I can't think of a real use of templates for resource manager.
This will be off the top of my head (not compiled or tested):

*** Source Snippet Removed ***
That's a very simple example of course, with no error checking and with minimal functionality. It does however demonstrate the technique I'm talking about: by parameterizing the cache class by resource type, you eliminate the need for down-casting when requesting a resource from the cache.

Yeah, I was using raw pointers. I will try to use smart pointers in my resource manager. Smart pointers overwhelmed me in the past because there are few of them in boost with small differences. Luckily, I already have boost installed (needed it for callbacks with bind/function combo). I will try to accomodate that source snippet of yours.

+rating for both of you of course

Share this post


Link to post
Share on other sites
Ok, so I'm actually in the process of writing an article for Gamedev.net about my resource manager, and your post

makes me regret that its not ready yet because I'll address all of your issues, and it would be nice to just point you at that article.

That said, here's the general layout of my system:

The core of the system is indeed a std::map between some resource-identifier and a shared_pointer to the resource class itself -- at least, that's how the client sees it. Both the identifier and the resource type are template parameters; there is no base resource type. The shared_pointer implimentation is also a template parameter for now, allowing the client to choose tr1::shared_pointer if their library supplies it (preferred) or any other conforming implimentation, such as the one supplied by the Boost library. I say "as the programmer sees it, because there is a meta-info class which is the *actual* map key, which hangs some extra load-time information on to the resource-identifier (which, right now, I used to hint to the manager whether this resource is intended to stay around forever, temporarily, or "just as long as we need it"). This is not a base-class either, it is also accomplished through template parameters and composistion -- beyond that you don't really need to care, since its an internal detail of the ResourceCache class. The only hint that its even there is the fact that you supply an enumeration during resource aquisision stating the intended lifetime of the resource.

This makes for a really nice and flexible system. If you don't want to create a large resource-identifier type (as you don't likely have a need to) you can just use std::string, supplying that as the appropriate template parameter. This works perfectly well for file-system resources, when you supply the path as identifier. You can also supply more symbolic identifier, such as an int or non-path string. The only requirement on what you choose is that it somehow can be mapped back to the resource in whatever storage system its located in. I'll talk about loading in a minute.

Now, the resource type itself has no base class, but my system allows for it to be a base class -- meaning that you can store resources of the same category, that are not necessarily the same type. You might load several different types of text-based resources (configuration data, scripts, dialog tables) into the same ResourceCache if you like. Of course, you do have to cast it appropriately once its returned to you.

Now lets talk about loading. In my system, loading is initiated through the ResourceCache, and at that time, a loader callback is supplied. This loader callback takes two parameters: a const reference to the key type, and a non-const reference to the resource type. So when the load is initiated through the ResourceCache, it goes ahead and creates a spot in the Cache for that resource object to occupy, and then calls on the callback to do the actual loading. Since you define and supply this loader callback, you can either use a literal key (which identifies the resource withing the storage system directly) or a symbolic key (which you could then look up in this loader callback to locate it in the storage system.)

This all sounds quite complex, but the code is only a couple hundred lines, and that's with extensive documentation and comments. Its really quite simple and intuitive to actually use.

The main things I set out to achieve are:
- to be entirely built upon standard C++ features.
- To be usable with any key and resource type without needing to derive from a base type.
- To support polymorphic resource types.
- To separate loading from the ResourceCache (thus making it more re-usable).
- To support different storage systems (filesystem, resources embedded in the executable, etc).


I'll leave it at that for now, please do feel free to ask me any questions you might have here, it will help me to address points of possible confusion in my article [grin]

Share this post


Link to post
Share on other sites
I saw that method of initializing resources (passing callbacks to ResourceManager) in some engine recently (can't say which).

There is one (I hope last) thing I am unsure of. Let's say I have my generic ResourceCache implemented, and I can specialise it into WhateverResourceINeedCache. Now, who should keep these caches?

Would it be better to make them all belong to a central AllResourceCaches class, or should TextureManager, MaterialManager all have their own caches?

Share this post


Link to post
Share on other sites
Even though you could achieve an ultra-generic AllResourceManager class that held textures, sound, level data, and everything else, it wouldn't really be a good idea, because, eventually, you have to be able to cast it to a type you can do something with.

I have usually separated it by resource type, TextureManager, SoundManager, ScriptManager, etc. and had these Manager classes owned by the Application class in my code.

An alternative is to have the Application class own "permanant" resources -- things that always need to be ready at any time, like GUI graphics and sounds -- while having all the non-permanent resources owned by a second manager of like type, which itself is owned by the level. This alternative configuration makes it super easy to ensure that all the per-level resources are gotten rid of when you are no longer in that level, at the cost of possibly, though unlikely, duplicating resources between the two. Its a matter of taste, and the system I've developed supports both options.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!