Jump to content
  • Advertisement
Sign in to follow this  
Nairou

Having trouble managing resources without duplicating them

This topic is 4398 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I'm designing my resource manager, and I've run into an issue of how to prevent resources within objects from being duplicated (which would defeat the purpose of having a resource manager). This problem may have been discussed before, but I didn't find it in a search, I'm hoping someone can show me what I'm overlooking here. I have my resource manager class, which keeps track of all of the resource files available and performs the loading of files into memory. Then I have my ResourceObject base class, which does the work of calling the resource manager to load the resource, and them parsing it into a usable state. This is subclassed into the various types of resources, such as Meshes, Textures, Sounds, etc., each of which inherit from this base class. So, for example, to load a new mesh all I would have to do is something like: Mesh& myMesh = new Mesh( "meshes/test.mesh" ); And the mesh file would be loaded and parsed by the mesh-specific code in the Mesh class, resulting in a ready-to-use object. The problem is in reference counting, what to do if another bit of code attempted to load the same resource again: Mesh& myOtherMesh = new Mesh( "meshes/test.mesh" ); Ideally, the resource manager would tell this new Mesh object that the data was already loaded, and the object would just reference the already-loaded data rather than loading and parsing the file all over again. The problem is the fact that the parsed resource data is currently stored within the Mesh object itself. The resource manager doesn't know the difference between the various types of resources, so it can't parse the file, only load it. It knows nothing about converting a mesh data file into a series of vertex buffers, that's why the individual resource classes exist. So then how would a central copy of the parsed resource be stored for each Mesh object to reference without storing their own individual copies of it? I hope this makes sense, ideas and opinions are greatly appreciated.

Share this post


Link to post
Share on other sites
Advertisement
You obviously have to keep track of things you've already loaded, and dish out pointers/references to them if someone asks for another one. Just have a std::map or similar of pointers of type ResourceObject in your manager, which you can look up based on the filename. Of course your resource manager has to be able to instantiate each of the resource types. This could be done using a template function. Just make sure all your resources only take the file name in their constructor, and a function like this might work.

template<class T>
const T* ResourceManager::Load<T>(string filename)
{
if (it's in our map of resources)
cast to type T and return
add new T(filename) to map
return resource
}


But that's all more complicated than it needs to be. We can do better. You don't need a central resource manager anyway, you want different types of resource treated seperately. I'm not advocating writing lots of resource managers, I'm advocating more templates.

Lets make a templated resource pointer class that we can use to wrap our resources. It acts basicly like a normal pointer. The only special thing it does is keep a static map of pointers to all the resources. I'll try to explain it by using it.

ResourcePtr<Model> myModel;
myModel.load("baddie3.model");
myModel->Render(location, blah);
//So it's used simply. But how does it let you avoid loading duplicates?
//Well, here's some pseudo code for the ResourcePtr itself.
template<class T>
class ResourcePtr
{
void load(string file)
{
if (in resource map)
currentResource = resource found in map
else
currentResource = new T(file);
add currentResource to map
}
T* operator->()
{
return currentResource;
}
T* currentResource; //Resource currently pointed to.
//This is static, so it's shared amoung all ResourcePtrs with the same template parameter.
static map<string, T*> resources;
};


Realistically you'd want to do reference counting or something, and I'm sure I've made mistakes, but I hope that gets the basic idea across.

Share this post


Link to post
Share on other sites
Very interesting! So basically you treat the ResourceObjects as unique and shared, one per file with pointers to them passed around as needed, and a second class that is actually used by the application to access those resources. That would definitely work, the idea is a bit more indirect and messy than I was trying for originally, but I don't yet see a way to keep it all in the ResourceObject and still have them unique and shared. You've definitely given me a lot more to think about though, thanks!

Share this post


Link to post
Share on other sites
That templated pointer idea is kind of cool! I hadn't thought of that.

My line of attack would be to have the resource manager hold a std::map<string,ResourceObject> (or std::map<string,ResourceObject*>, which I suppose is more likely) and just cache any resource which has been loaded this level. If you have an extended world (so level loading/unloading doesn't happen very often) you may need to implement reference counting or some sort of priority cache for unloaded objects.

Share this post


Link to post
Share on other sites
Quote:
Original post by Bob Janova
If you have an extended world (so level loading/unloading doesn't happen very often) you may need to implement reference counting or some sort of priority cache for unloaded objects.


Maybe an LRU cache.

My approach is similar but slightly different, I use a factory called ResourceManager to load my resources transparently. Putting the resource into a reference would likely not make much sense for me in Java anyway. So rather than ResourcePtr I have something like Model ship = res.getModel("myship") which returns a cached reference or loads the model from disk, also grabbing the textures, materials, etc specified by the model from cache if they are already loaded. If something is not in cache, a new instance is created, and the constructor of each resource takes care of parsing the resource file itself. So the ResourceManager still doesn't care about the contents of the resource files.

But, compared to the template approach possible with C++, I need to write new code to support each new kind of resource. It was also possible to combine the caching into one cache by using a superclass, but having one big cache may not be very performance friendly.

Share this post


Link to post
Share on other sites
Quote:
Original post by lightbringer
My approach is similar but slightly different, I use a factory called ResourceManager to load my resources transparently. Putting the resource into a reference would likely not make much sense for me in Java anyway. So rather than ResourcePtr I have something like Model ship = res.getModel("myship") which returns a cached reference or loads the model from disk, also grabbing the textures, materials, etc specified by the model from cache if they are already loaded. If something is not in cache, a new instance is created, and the constructor of each resource takes care of parsing the resource file itself. So the ResourceManager still doesn't care about the contents of the resource files.


Yes this was the other idea I had as well, just calling the resource manager directly to return a reference to the ResourceObject. Being able to deal directly with the resource objects is something I'm striving for, as opposed to accessing them indirectly through ResourcePtr's as in the first solution. But it still has the problem of requiring the resource manager to be aware of every type of resource in order to provide a new or cached copy of it to the caller.

Ideally I want to find a way to hide the caching in the background, so that I can simply do something like:

Mesh myMesh( "monster.mesh" );

and the Mesh object will quietly access the resource manager to either load a new file or use a cached copy. But the very idea of caching seems to imply a second object for the resource manager to manage and return to the Mesh object, as in the first ResourcePtr example, and once you have two objects you either remove yourself from directly using the ResourceObjects or you start duplicating efforts between the two object copies. I'm still trying to find a solution that accomplishes both caching and direct object access.

Share this post


Link to post
Share on other sites
(On a side note, since I'm more used to Java, when I say reference, I probably mean pointer. Or not. In any case, the idea is to pass around objects "byref" and not "byval")

Well, the ResourceManager only has to be aware of every type of resource if you want to have separate caches for each type (I think this is preferable).

I'm not exactly sure about what you mean by a second object, however. The templated example was a wrapper object, but with any direct kind of caching scheme there is no duplicate object involved. The only thing to watch out for is to remember to call the factory method instead of constructing a new object.

The word "copy" here is misleading, since you get back a reference to an already existing object. So if I call res.getModel("myship") twice, I get two references to the same model, and if I modify one of them, I am of course modifying the other. So the resource caching scheme is not directly suitable for real-time mesh deformation and such, as you would have to create a deep copy of the mesh resource first.

If you want transparent resource caching while only calling the resource class, then that is still possible without templates, if every resource class implements its own caching by having the cache be a private static class member and providing a factory method. You save yourself the ResourceManager, but now have your resource caching code spread all over the place, which may not be what you want.

The real problem with the static cache approach, but I think it is the same for the parameterized approach since the idea is essentially the same, is when you want to clear all caches at once (for instance when loading a new level). If you have no centralized controller for the caches, you will have to remember to call each resource class's clearCache() method. In the case of a ResourceManager you abstract such details away into a clearAllCaches() method and never have to think about it again (until you add a new resource type).

Share this post


Link to post
Share on other sites
Nairou mentioned abstracting the resource system away entirely. I actually do that to a large extent. Keeping with the model example, I have a ModelData class (of a different name) which is the actual resource I wrap with ResourcePtr. But then I have another layer, the Model class itself. This model keeps all the data that isn't shared between instances of a model, things like location and animation frame and so on, as well as a ResourcePtr to the model data class with it's geometry. So the fact that the models are sharing data using ResourcePtrs is behind the scenes.

Oh, you guys probably realise this but just in case: Make sure that things using shared resources have read only access, i.e. a const pointer.

Share this post


Link to post
Share on other sites
Good point, Resource data should really be immutable if cached. I also do the same thing you mention (who doesn't :)) by wrapping the Model in an Entity (or Geometry, or whatever else) class which lives in my scene graph. The Model is the immutable visual representation, the Entity a specific instance.

Share this post


Link to post
Share on other sites
Nairou: the resource manager doesn't need to know what type of thing a resource is once it's successfully loaded it once.
class ResourceManager {
std::map <string,Resource*> cache;
...
public Resource* Get(string name){
Resource* res = cache[name];
if(res == null){
res = Load(name);
cache[name] = res;
}
return res;
}

public Resource* Load(string name){
// some magic here to load and create a resource
}
}


The Get function doesn't need to know what's in the cache.

As for Mesh thing(name), I don't know if you're allowed to do this in C++:
class Mesh {
Mesh(string name){ this = (Mesh)ResourceManager.Get(name); }
}


Personally, I'd be happy writing Mesh thing = ResourceManager.Get("mymesh.x") anyway.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!