Jump to content
  • Advertisement
  • 09/30/14 08:08 PM
    Sign in to follow this  

    A Resource Manager for Game Assets

    General and Gameplay Programming

    TheItalianJob71

    Preamble

    This article has been written based on my personal experience, if you think you are offended in some ways because of my personal opinions about programming and / or my coding style, please stop reading now. I am not a guru programmer and I don't want to impose anything on anyone.

    Introduction

    During the development of my engine(s), I have always had the need for better handling of game assets, even if in the past it was sufficient to hard code the needed resources at startup. Now with the added speed of modern computers it is possible to create something more advanced, dynamic and general-purpose. The most important points I needed for a resource manager were:
    • reference counted resources.
    • If a resource is already present in the database, expose it as raw pointer, else load it.
    • know automatically when to free the resource.
    • fast access using string to retrieve resources
    • clean everything on exit without manually deleting the resources.
    • mapping resource using a referenced structure.
    At this point it looks like I am going to reinvent the wheel, given the assumption that smart pointers are now included with the new C++ standard. My personal problem with the smart pointers is that it was very difficult to create the kind of data structure I wanted. My idea was to create an unordered map of shared pointers and then hand out a weak pointer when the resource was asked for. The problem with this data organization is that the weak pointer basically 'observes' the resource but doesn't hold it. On the contrary using an unordered map of weak pointers and handing out a shared pointer created different problems, which could be resolved using a custom deleter and other indirect strategies. One of this was that if I loaded the resource for the first time, its count was set to 1 and when the same resource was asked for its count was increased to 2. So I still had the problem to ignore the first reference, which was again solvable using a custom deleter and tracking how many resources where effectively still left. Further more, shared pointers are thread safe and according to the standard they are synced even if there is no strict necessity. I am not saying that smart pointer are useless, they are used quite extensively, but my approach needed something different; I needed to have total control of reference counting. Basically what I needed was a wrapper class to store the pointer to the loaded resource, and its reference counting. Note that the code I copied from my engine uses some other utility functions - they are not necessary, because they handle error messaging and string 'normalisation' (eliminating white spacing and lowering the string down), so you can easily ignore those functions and substitute them with yours. I have also removed all debugging printing during the execution, to keep things clearer. Let's have a look at the base class for resources. class CResourceBase { template < class T > friend class CResourceManager; private: int References; void IncReferences() { References++; } void DecReferences() { References--; } protected: // copy constructor and = operator are kept private CResourceBase(const CResourceBase& object) { } CResourceBase& operator=(const CResourceBase& object) { return *this; } // resource filename std::string ResourceFileName; public: const std::string &GetResourceFileName() const { return ResourceFileName; } const int GetReferencesCount() const { return References; } //////////////////////////////////////////////// // ctor / dtor CResourceBase( const std::string& resourcefilename ,void *args ) { // exit with an error if filename is empty if ( resourcefilename.empty() ) CMessage::Error("Empty filename not allowed"); // init data members References = 0; ResourceFileName=CStringFormatter::TrimAndLower( resourcefilename ); } virtual ~CResourceBase() { } }; The class is self-explanatory - the interesting point here is the constructor, which needs a resource filename, including the full path of your resource and a void pointer to an argument class in case you wanted to load the resource with some initial default parameters. The args pointer comes in handy when you want to instantiate assets during runtime and don't want to load them. There are some cases where this is useful, for every other case the constructor will serve our purposes well. All of our assets will inherit from this class. There is, obviously, the reference counter and some functions for accessing it.

    The Resource Manager

    The resource manager basically is a wrapper for an unordered map. It uses the string as a key and maps it to a raw pointer. I have decided to use an unordered map because I don't need a sorted arrangement of assets, I really do care at retrieving them in the fastest possible way, in fact accessing a map is O(log N), while for an unordered map is O(N). In addition just because the unordered map is constant speed (O(1)) doesn't mean that it's faster than a map (of order log(N)). Anyway, in my test cases the N value wasn't so huge, so the unordered map has always been faster than map, thus I decided to use this particular data structure as the base of my resource manager. The most important functions in the resource map are Load and Unload. The Load function tries to retrieve the assets from the unordered map. If the asset is present, its reference count is increased and the wrapped pointer is returned. If it's not found in the database, the function creates a new asset class, increases its reference count, stores it in the map and returns its wrapped pointer. Note that its the programmer's responsibility to create an asset class with a proper constructor. The base class, inherited from CResourceBase class, must provide a string containing the full path from where the asset needs to be loaded and an argument class if any - this will be clearer when the example is provided. The Unload function does exactly the opposite: looks for the requested asset given its file name, if the resource is found, its reference counter is decreased, and if it reaches zero the associated memory is released. Since I think that a good programmer understands better 1000 lines of code rather than 1000 lines of words, here you have the entire resource manager: template < class T > class CResourceManager { private: // data members std::unordered_map< std::string, T* > Map; std::string Name; // copy constructor and = operator are kept private CResourceManager(const CResourceManager&) { }; CResourceManager &operator = (const CResourceManager& ) { return *this; } // force removal for each node void ReleaseAll() { std::unordered_map< std::string, T* >::iterator it=Map.begin(); while ( it!=Map.end() ) { delete (*it).second; it=Map.erase( it ); } } public: /////////////////////////////////////////////////// // add an asset to the database T *Load( const std::string &filename, void *args ) { // check if filename is not empty if ( filename.empty() ) CMessage::Error("filename cannot be null"); // normalize it std::string FileName=CStringFormatter::TrimAndLower( filename ); // looks in the map to see if the // resource is already loaded std::unordered_map< std::string, T* >::iterator it = Map.find( FileName ); if (it != Map.end()) { (*it).second->IncReferences(); return (*it).second; } // if we get here the resource must be loaded // allocate new resource using the raii paradigm // you must supply the class with a proper constructor // see header for details T *resource= new T( FileName, args ); // increase references , this sets the references count to 1 resource->IncReferences(); // insert into the map Map.insert( std::pair< std::string, T* > ( FileName, resource ) ); return resource; } /////////////////////////////////////////////////////////// // deleting an item bool Unload ( const std::string &filename ) { // check if filename is not empty if ( filename.empty() ) CMessage::Error("filename cannot be null"); // normalize it std::string FileName=CStringFormatter::TrimAndLower( filename ); // find the item to delete std::unordered_map< std::string, T* >::iterator it = Map.find( FileName ); if (it != Map.end()) { // decrease references (*it).second->DecReferences(); // if item has 0 references, means // the item isn't more used so , // delete from main database if ( (*it).second->GetReferencesCount()==0 ) { // call the destructor delete( (*it).second ); Map.erase( it ); } return true; } CMessage::Error("cannot find %s\n",FileName.c_str()); return false; } ////////////////////////////////////////////////////////////////////// // initialise void Initialise( const std::string &name ) { // check if name is not empty if ( name.empty() ) CMessage::Error("Null name is not allowed"); // normalize it Name=CStringFormatter::TrimAndLower( name ); } //////////////////////////////////////////////// // get name for database const std::string &GetName() const { return Name; } const int Size() const { return Map.size(); } /////////////////////////////////////////////// // ctor / dtor CResourceManager() { } ~CResourceManager() { ReleaseAll(); } };

    Mapping Resources

    The resource manager presented here is fully functional of its own, but we want to be able to use assets inside a game object represented by a class. Think about a 3D object, which is made of different 3D meshes, combined together in a sort of hierarchial structure, like a simple robot arm, makes the idea clearer. The object is composed of simple building blocks, like cubes and cylinders. we want to reuse every object as much as possible and also we want to access them quickly, in case we want to rotate a single joint. The engine must fetch the object quickly, without any brute force approach, also we want a name for the asset so we can address it using human readable names, which are easier to remember and to organize. The idea is to write a resource mapper which uses another unordered map using strings as keys and addresses from the resource database as the mapped value. We need also to specify if we want to allow the asset to be present multiple times or not. The reason behind this is simple - think again at the 3D robot arm. We need to use multiple times a cube for example, but if we use the same resource mapper for a shader, we need to keep each of the shaders only once. Everything will become clearer as the code for the mapper unfolds further ahead. template < class T > class CResourceMap { private: ///////////////////////////////////////////////////////// // find in all the map the value requested bool IsValNonUnique( const std::string &filename ) { // if duplicates are allowed , then return alwasy true if ( Duplicates ) return true; // else , check if element by value is already present // if it is found, then rturn treu, else exit with false std::unordered_map< std::string, T* >::iterator it= Map.begin(); while( it != Map.end() ) { if ( ( it->second->GetResourceFileName() == filename ) ) return false; ++it; } return true; } ////////////////////////////////////////////////////////////////////////////// // private data std::string Name; // name for this resource mapper int Verbose; // flag for debugging messages int Duplicates; // allows or disallwos duplicated filenames for resources CResourceManager *ResourceManager; // attached resource manager std::unordered_map< std::string, T* > Map; // resource mapper // copy constructor and = operator are kept private CResourceMap(const CResourceMap&) { }; CResourceMap &operator = (const CResourceMap& ) { return *this; } public: ////////////////////////////////////////////////////////////////////////////////////// // adds a new element T *Add( const std::string &resourcename,const std::string &filename,void *args=0 ) { if ( ResourceManager==NULL ) CMessage::Error("DataBase cannot be NULL (5)" ); if ( filename.empty() ) CMessage::Error("%s : filename cannot be null",Name.c_str()); if ( resourcename.empty() ) CMessage::Error("%s : resourcename cannot be null",Name.c_str()); std::string ResourceName=CStringFormatter::TrimAndLower( resourcename ); // looks in the hashmap to see if the // resource is already loaded std::unordered_map< std::string, T* >::iterator it = Map.find( ResourceName ); if ( it==Map.end() ) { std::string FileName=CStringFormatter::TrimAndLower( filename ); // if duplicates flag is set to true , duplicated mapped values // are allowed, if duplicates flas is set to false, duplicates won't be allowed if ( IsValNonUnique( FileName ) ) { T *resource=ResourceManager->Load( FileName,args ); // allocate new resource using the raii paradigm Map.insert( std::pair< std::string, T* > ( ResourceName, resource ) ); return resource; } else { // if we get here and duplicates flag is set to false // the filename id duplicated CMessage::Error("Filename name %s must be unique\n",FileName.c_str() ); } } // if we get here means that resource name is duplicated CMessage::Error("Resource name %s must be unique\n",ResourceName.c_str() ); return nullptr; } ///////////////////////////////////////////////////////// // delete element using resourcename bool Remove( const std::string &resourcename ) { if ( ResourceManager==NULL ) CMessage::Error("DataBase cannot be NULL (4)"); if ( resourcename.empty() ) CMessage::Error("%s : resourcename cannot be null",Name.c_str()); std::string ResourceName=CStringFormatter::TrimAndLower( resourcename ); if ( Verbose ) CMessage::Trace("%-64s: Removal proposal for : %s\n",Name.c_str(),ResourceName.c_str() ); // do we have this item ? std::unordered_map< std::string, T* >::iterator it = Map.find( ResourceName ); // yes, delete element, since it is a reference counted pointer, // the reference count will be decreased if ( it != Map.end() ) { // save resource name std::string filename=(*it).second->GetResourceFileName(); // erase from this map Map.erase ( it ); // check if it is unique and erase it eventually ResourceManager->Unload( filename ); return true; } // if we get here , node couldn't be found // so , exit with an error CMessage::Error("%s : couldn't delete %s\n",Name.c_str(), ResourceName.c_str() ); return false; } ////////////////////////////////////////////////////////// // clear all elements from map void Clear() { std::unordered_map< std::string, T* >::iterator it=Map.begin(); // walk trhough all the map while ( it!=Map.end() ) { // save resource name std::string filename=(*it).second->GetResourceFileName(); // clear from this map it=Map.erase ( it ); // check if it is unique and erase it eventually ResourceManager->Unload( filename ); } } ////////////////////////////////////////////////////////// // dummps database content to a string std::string Dump() { if ( ResourceManager==NULL ) CMessage::Error("DataBase cannot be NULL (3)"); std::string str=CStringFormatter::Format("\nDumping database %s\n\n",Name.c_str() ); for ( std::unordered_map< std::string, T* >::iterator it = Map.begin(); it != Map.end(); ++it ) { str+=CStringFormatter::Format("resourcename : %s , %s\n", (*it).first.c_str(), (*it).second->GetResourceFileName().c_str() ); } return str; } ///////////////////////////////////////////////////////// // getters ///////////////////////////////////////////////////////// // gets arrays name const std::string &GetName() const { return Name; } const int Size() const { return Map.size(); } ////////////////////////////////////////////////////////// // gets const reference to resource manager const CResourceManager *GetResourceManager() { return ResourceManager; } ///////////////////////////////////////////////////////// // gets element using resourcename, you should use this // as a debug feature or to get shared pointer and later // use it , using it in a section where performance is // needed might slow down things a bit T *Get( const std::string &resourcename ) { if ( ResourceManager==NULL ) CMessage::Error("DataBase cannot be NULL (2)"); if ( resourcename.empty() ) CMessage::Error("%s : resourcename cannot be null",Name.c_str()); std::string ResourceName=CStringFormatter::TrimAndLower( resourcename ); std::unordered_map< std::string, T* >::iterator it; if ( Verbose ) { CMessage::Trace("%-64s: %s\n",Name.c_str(),CStringFormatter::Format("Looking for %s",ResourceName.c_str() ).c_str()); } // do we have this item ? it = Map.find( ResourceName ); // yes, return pointer to element if ( it != Map.end() ) return it->second; // if we get here , node couldn't be found thus , exit with a throw CMessage::Error("%s : couldn't find %s",Name.c_str(), ResourceName.c_str() ); // this point is never reached in case of failure return nullptr; } ///////////////////////////////////////////////////////// // setters void AllowDuplicates() { Duplicates=true; } void DisallowDuplicates() { Duplicates=false; } void SetVerbose() { Verbose=true; } void SetQuiet() { Verbose=false; } //////////////////////////////////////////////////////////// // initialise resource mapper void Initialise( const std::string &name, CResourceManager *resourcemanager, bool verbose,bool duplicates ) { if ( resourcemanager==NULL ) CMessage::Error("DataBase cannot be NULL 1"); if ( name.empty() ) CMessage::Error("Array name cannot be null"); Name=CStringFormatter::TrimAndLower( name ); // normalized name string ResourceManager=resourcemanager; // copy manager pointer // setting up verbose or quiet mode Verbose=verbose; // setting up allowing or disallowing duplicates Duplicates=duplicates; // emit debug info if ( Verbose ) { if ( Duplicates ) CMessage::Trace("%-64s: Allows duplicates\n",Name.c_str() ); else if ( !Duplicates ) CMessage::Trace("%-64s: Disallows duplicates\n",Name.c_str() ); } } ///////////////////////////////////////////////////////// // ctor / dtor CResourceMap() { Verbose=-1; // undetermined state Duplicates=-1; // undetermined state ResourceManager=NULL; // no resource manager assigned } ~CResourceMap() { if ( Verbose ) CMessage::Trace("%-64s: Releasing\n",Name.c_str() ); Clear(); // remove elements if unique } }; } Basically, the class is a wrapper for the resource database operations. Among the private data, as you can see, its present an unordered map where the first key is a string and the mapped value is the pointer directly mapped from the resource database. Let's have a look at the function members now. The Add function performs many tasks. First, checks if the name for the asset is already present, since duplicated names for the assets are not allowed. If name is not present, it performs the attempt to upload the assets from the resource database, then it checks if the filename is unique and if the duplicates flag is not set to true. Here I have used a brute force approach, the reason behind it is that if I wanted to have a sort of bidirectional mapping, I should have used a more complex data structure, but I wanted to keep things simple and stupid. At this point the resource database uploads the asset, and if it's present, it hands back immediately the address for the required resource. If not it loads it, making the process completely transparent for the resource mapper and it gets stored in the unsorted map data structure. Note again, that all the error checking are just wrappers for a throw, you may want to replace with your error checking code, without compromising the Add function itself. The Remove function is a little bit more interesting, basically the safety checks are the same used in Add, the resource is erased from the map, the resource database removal function is invoked, but the resource database doesn't destroy it if it is still shared in some other places. By 'some other places' I mean that the asset may be still be present in the same resource mapper or in another resource mapper instantiated somewhere else in your game. This will be clearer with a practical example further ahead. The Clear function basically performs the erasure of the entire resource map, using the same counted reference mechanism from the resource database. The Get function retrieves the named resource by specifing its resource name and gives back the resource pointer. The Initialise function attaches the resource mapper to the resource database.

    Example of Usage

    First of all, we need a base class, which could be a game object. Let's call it foo just for example class CFoo : public vml::CResourceBase { //////////////////////////////////////////////////// // copy constructor is private // no copies allowed since classes // are referenced CFoo( const CFoo &foo ) : CResourceBase ( foo ) { } //////////////////////////////////////////////////// // overload operator is private, // no copies allowed since classes // are referenced CFoo &operator =( CFoo &foo ) { if ( this==&foo ) return *this; return *this; } public: //////////////////////////////////////////////// // ctor / dtor // this constructor must be present CFoo(const std::string &resourcefilename, void *args ) : CResourceBase( resourcefilename,args ) { } // regular base constructor and destructor Cfoo() {} ~CFoo() { } }; Now we can instantiate our resource database and resource mappers. CResourceManager rm; CResourceMap mymap1; CResourceMap mymap2; I have createed a resource manager and two resource mappers here. // create a resource database rm.Initialise("FooDatabase", vml::CResourceManager::VERBOSE ); // attach this database to the resource mappers, bot of them allowd duplicates mymap1.Initialise( "foolist1",&rm, true,true ); mymap2.Initialise( "foolist2",&rm, true,true ); // populate first resoruce mapper // the '0' argument means that the resource 'a' , whose filename is foo1.txt // doesn't take any additional values at construction time mymap1.Add( "a","foo1.txt",0 ); mymap1.Add( "b","foo1.txt",0 ); mymap1.Add( "c","foo2.txt",0 ); mymap1.Add( "d","foo2.txt",0 ); mymap1.Add( "e","foo1.txt",0 ); mymap1.Add( "f","foo1.txt",0 ); mymap1.Add( "g","foo3.txt",0 ); // populate second resource mapper mymap2.Add( "a","foo3.txt",0 ); mymap2.Add( "b","foo1.txt",0 ); mymap2.Add( "c","foo3.txt",0 ); mymap2.Add( "d","foo1.txt",0 ); mymap2.Add( "e","foo2.txt",0 ); mymap2.Add( "f","foo1.txt",0 ); mymap2.Add( "g","foo2.txt",0 ); // dump content into a stl string which can be printed as you like std::string text=rm.Dump(); Running this example and printing the text content gives: Dumping database foodatabase Filename : foo1.txt , references : 7 Filename : foo2.txt , references : 4 Filename : foo3.txt , references : 3 This concludes the article. I hope it will be useful for you, thanks for reading.


      Report Article
    Sign in to follow this  


    User Feedback




    I will chime-in and say I am happy at least to see it was not implemented as a Singleton.

    Share this comment


    Link to comment
    Share on other sites

    Singleton is one of the biggest problems with 'manager' classes, so the author has done well to avoid it. I'll raise your chime-in to say that relying on a resource base class is intrusive, heavy, and brittle, and that the code could be simplified by using C++11 smart pointers to handle lifetime management and ownership.

     

    I had a pretty nice pre-C++11 resource manager that used boost's shared pointer, that I had always meant to write an article on. Maybe I should revisit that code and update to make use of useful C++11/C++14 features. I can think of a handful that would make for real slick interface/usage.

    Share this comment


    Link to comment
    Share on other sites

    I think you can use std::shared_ptr for this. Use use_count() method to check number of references. When the use_count is 1 then your resource is referenced only by the resource manager.

    Share this comment


    Link to comment
    Share on other sites

    Nightcreature, the resource manager uses the IncReferences and DecReferences, which are private members, Majo33, about the use_count, that is the problem i wanted to avoid , using the shared pointer i were in the condition that no resource mapper had the 'ownership' for that asset , but it was still loaded in memory. I tried to overcome the problem, but the code started to look unnecessary complicated.

    About the c++11 features , maybe will rewrite the code to use these new features one day.

    Share this comment


    Link to comment
    Share on other sites

    When you start using features like friend it generally means you designed your interface wrong and should rethink your strategy of doing this.

    Share this comment


    Link to comment
    Share on other sites

    I,m using seperate level & global data myself.

    Global : used in each level, will be cleaned on exit game.

    Level : for only 1 level, will be cleaned after exit level.

    greets

    Share this comment


    Link to comment
    Share on other sites

    ...I also hate to be that guy but the article image uses copywritten artwork from Hyperbole and a Half.

     

    Seems to have been fixed.

    Share this comment


    Link to comment
    Share on other sites

    >> using the shared pointer i were in the condition that no resource mapper had the 'ownership' for that asset , but it was still loaded in memory.

     

    But that's the behavior you probably want. If someone other than any of the managers holds a reference count (e.g. a shared_ptr), then it should be because they have an interest in the lifetime of the object it points to -- thus, the object *should* remain in memory because, for example, the code that holds the reference might try to render that resource. It shouldn't die just because none of the managers know about it any more.

     

    You have two classes of entities that have an interest in the lifetime of the object -- you have users of the resource that need the thing to be alive as long as they need, regardless of caching; and you have the manager of the resource who's sole interest is caching to share resources, eliminate redundancy, and (optionally) to extend the lifetime of a resource that currently has no users (e.g. keep the thing alive while there's no urgent need to release it, because you think the thing might be needed soon). So, both users and the manager are owners.

     

    The only minor wrinkle is that you may deem it undesirable for there to be multiple copies of an asset in memory, such as when the manager forgets it but it's still being used by somebody, and then another somebody needs that asset later, causing a new, distinct copy to be loaded through a manager. This is sub-optimal, perhaps, but correct. Any pathological, worst-case program behavior that might result from this property should be considered a logic error on the part of client code, although there are ways even for the manager to release its ownership interest in the asset while still being able to find it again -- e.g. a 'victim cache'.

    Share this comment


    Link to comment
    Share on other sites

    I think you can use std::shared_ptr for this. Use use_count() method to check number of references. When the use_count is 1 then your resource is referenced only by the resource manager.

     
    Alternatively you can have your resource manager store std::weak_ptrs, and call std::weak_ptr::lock() or check std::weak_ptr::expired(). If weak_ptr failed to lock or was expired, then it means it was de-allocated because of zero references (the resource manager's weak_ptr not counting as a reference), so you can then re-load it.

     

    Basically:

    typedef TextureKey; //ID, filepath, whatever.
    std::unordered_map<TextureKey, std::weak_ptr<Texture>> textures;
    
    std::shared_ptr<Texture> GetTexture(TextureKey key)
    {
         //Get the pointer to the texture, automatically creating a null pointer if it doesn't yet exist.
         std::weak_ptr<Texture> &texturePtr = this->textures[key]; 
        
         //If the texture is currently loaded...
         std::shared_ptr<Texture> loadedTexture = texturePtr.lock();
         if(loadedTexture)
         {
              //...return it.
              return loadedTexture;
         }
         
         //Otherwise, load the texture.
         loadedTexture = std::make_shared<Texture>(GetCreationDetails(key));
    
         //Set the map's pointer to the new texture.
         texturePtr = loadedTexture;
    
         //Return the pointer.
         return loadedTexture;
    }

    It automatically frees the resource if there are no longer any living references to it (apart from the manager's reference), and loads it again the next time it is requested.

     

    This could actually even be wrapped up as a single generic template function rather than a templated class or class hierarchy.

    Share this comment


    Link to comment
    Share on other sites

    One minor wrinkle with with std::weak_ptr referring to a std::shared_ptr made with std::make_shared is that make_shared attempts to optimize spacial locality by combining the object with the control block (the ref-counts and such) for objects up to a certain size (I think a cache-line in total, so 64 bytes minus the size of the control-block itself on most platforms). std::weak_ptr normally would not preserve the object allocation, but it does preserve the control block allocation, thus when they are combined, std::weak_ptr can actually cause the object (albeit a small one) to remain in memory even after no std::shared_ptr holds a reference count. You might or might not consider this to be a problem, but its definitely something to be aware of.

     

    [Note -- leaving this for context, but on closer examination it appears to be incorrect]

    If the small object represents a costlier resource (e.g. the small object owns a sizable allocation, device handle, etc) this can become troublesome.

     

    You can prevent this from happening by not using std::make_shared, but you'll give up the locality optimization and pay the normal pointer-hop to dereference the object each time you access it, and likely pay a cache hit.

    Share this comment


    Link to comment
    Share on other sites

    That's very interesting. You're saying that it doesn't just keep the raw bytes of the 'small object' in memory (which is fine), but it also never calls the destructor on it, until even after all the weak_ptrs go out of scope (which is terrible)?

     

    That's pretty bad. I can't replicate that negative behavior with GCC 4.8.1 - here's my test code. Maybe that only occurs with plain-old-data (not sure how to test for that)? I would imagine they'd call placement new (during construction) and just directly call the destructor when the shared references equal zero.

     

    Here's a better example, just in-case std::shared_ptr::reset() has different behavior.

    Share this comment


    Link to comment
    Share on other sites

    Question to nightcreature83, I wanted to keep the reference increment and decrement functions private but accessible from the resource manager, preventing the user to artificially modify the reference counting.

    Do you know of a better way to achieve this without frinedship?? just curious.

    Share this comment


    Link to comment
    Share on other sites

    Servant -- Actually, I revise my statement. I'm not certain but you're probably right that the destructor is at least called. Only the "small object" portion would remain in that case. Far less troublesome, but still irksome.

     

    ItalianJob -- You'll notice that shared_ptr doesn't have exposed increment and decrement operations at all -- its all implicitly managed during construction/deconstruction, copy, move, and other special operations like reset(). Through the exposed interface, you can look at the refcount but you can't touch. This is an example of 'maintaining class invariants'. This is the essential function of shared_ptr -- the only other thing it does is hold a pointer to some object and provide a means to hold and then call a special deallocation function if needed.

     

    There's no good way to avoid friendship with the design you have, the better option is to design it out. In the end, the design you have either requires inheritence combined with the friendship system you have, or devout following of the rule of 3/rule of 5 implementing reference counting semantics. None of these are great options, such inheritance patterns invade code that ought to be able to ignore that it has a ref count component, and worse, you can't use your resource manager with pre-existing classes. The "proper" solution is to invert the relationship, and then you end up with shared_ptr, essentially -- shared_ptr and friends are basically just boxes you can put things in to take care of lifetime management.

     

    In general, you should always be suspicious of 'friend' relationships -- friend relationships in C++ are actually more tightly coupled than inheritance relationships, it breaks encapsulation entirely (no contract can be enforced upon the friend, its 100% trust, unlike inheritance) and forces the 'base' class (the one from which friendship is declared) to know about (and be modified to suit) any class that would need access to its internals. This is why I called this arrangement "intrusive, heavy, and brittle".

     

    Even regular old inheritance, when not used strictly to express an is-a relationship, is suspicious -- less so than friendship -- but suspicious enough to take a careful look at.

    Share this comment


    Link to comment
    Share on other sites

    Alternatively you can have your resource manager store std::weak_ptrs, and call std::weak_ptr::lock() or check std::weak_ptr::expired(). If weak_ptr failed to lock or was expired, then it means it was de-allocated because of zero references (the resource manager's weak_ptr not counting as a reference), so you can then re-load it.

     

    /..../

     

    This could actually even be wrapped up as a single generic template function rather than a templated class or class hierarchy.

     

     

    I've done exactly this. Works pretty well.

    https://github.com/madeso/euphoria/blob/master/euphoria/cache.h

     

    I noticed however that the create function probably doesn't need to be a parameter, should change that...

    Share this comment


    Link to comment
    Share on other sites
    When working on a game we have complete control on the resources we use. We can choose the file format s, the methods used to identify the various resources and how to represent relationships between them. We are not limited by external constraints like in other softwares. Moreover, resources are usually loaded and unloaded at very specific moment of time. We can indeed identify global, level or frame specific resources. There is really no need of fancy and abstract resource managers.  
     
    By only stating your objectives and no motivation behind them it seem you are trying to solve an abstract and non-existent problem. In my opinion, the first thing an article like this should do is presenting the problem and motivating the decisions made in the implementation and design of the solution.
    For example:
    1. Why do you think you need reference counting? Have you considered other alternatives? How bad is resource duplication and late deallocation is in your system? Have you considered the possibility a resource is unloaded just before it is requested again?
    2. Why you think you can't control when a resource is deallocated and you need an automatic system to do this? For example, if you have resources that should be alive for an entire level, you may decide you can simply deallocate all of them at the end of the frame.
    3. Why do you think you need strings to access your resources? String maps are not fast and resource names often differ only in small parts. Have you tried other alternatives (like using IDs)?.
    4. Why do you need a map to maintain the relationships between resources? Why resources can't do it by themselves?

    Share this comment


    Link to comment
    Share on other sites

    I noticed however that the create function probably doesn't need to be a parameter, should change that...

     

    I agree it should either be a parameter, or a templated or overloadable function. But perhaps the default version of the templated function should pass the key and settings directly into the constructor of the object.

    Share this comment


    Link to comment
    Share on other sites

    Singleton is one of the biggest problems with 'manager' classes, so the author has done well to avoid it. I'll raise your chime-in to say that relying on a resource base class is intrusive, heavy, and brittle, and that the code could be simplified by using C++11 smart pointers to handle lifetime management and ownership.

     

    I will chime-in and say I am happy at least to see it was not implemented as a Singleton.

     

    I'm curious.... why the anti-singleton sentiment here? It would make plenty of sense to use it for something like this in that one single, global instance is pretty much what you'd want from a resource manager!

    Share this comment


    Link to comment
    Share on other sites

    It would make plenty of sense to use it for something like this in that one single, global instance is pretty much what you'd want from a resource manager!

     

    Your assumption is one of the biggest reasons (outside of the technical/philosophical reasons) why a singleton is a bad idea. In the case of a resource manager, the user (developer) may want to have one for UI elements and another for per-level resources, for example.

     

    Many cases of "there can be only one" are really more like "I personally don't see a need for more than one."

    Share this comment


    Link to comment
    Share on other sites

    I'm curious.... why the anti-singleton sentiment here? It would make plenty of sense to use it for something like this in that one single, global instance is pretty much what you'd want from a resource manager!

     

    A single global instance? Fine. Make it a global. Or a global unique_ptr if you need to delay its construction until after main() is entered.

     

    But enforcing a single instance of it for no reason? Singletons are for putting a lock on an object preventing it from being constructed more than once.

    Singletons don't say, "I only need one.", singletons say, "Make absolutely sure that there can only ever be one."

     

    Singletons enforce that there is a 'single' instance. Some things are genuinely useful as globals: Like logging. But that doesn't mean you want to enforce that there is only one logging instance - you might want separate instances of loggers for separate subsystems (for example).

     

    std::cout and std::cin are both globals. Neither are singletons, which is good. What if you want a separate std::cout for error output instead of just general output? Well, create another global called std::cerr, and... oh wait, they already did. If std::cout was a singleton, std::cerr couldn't exist (unless they copy+pasted the code of that singleton, instead of just creating a second instance of it).

     

    The problem is not really truly with singletons (there are very few situations where they are useful, but those rare situations do exist), the problem is most people abuse singletons when they really just want a global (whether initialized before or after main() enters). A secondary semi-related issue is over-using (non-const) globals out of laziness or poor design (or genuine lack of time when being rushed on a project), but that wasn't being commented on.

    Share this comment


    Link to comment
    Share on other sites

     

    It would make plenty of sense to use it for something like this in that one single, global instance is pretty much what you'd want from a resource manager!

     

    Your assumption is one of the biggest reasons (outside of the technical/philosophical reasons) why a singleton is a bad idea. In the case of a resource manager, the user (developer) may want to have one for UI elements and another for per-level resources, for example.

     

    Many cases of "there can be only one" are really more like "I personally don't see a need for more than one."

     

     

    Well it's not really an assumption as I've had plenty of experience in implementing various systems and I just go with what has worked best (for me). Clearly it's a matter of personal tastes. I like the idea of a single, unified, global instance of a system that has a set responsibility BUT there are times when multiple instances of a system can use quite useful too. Mix and match! 

     

    So how would multiple instances of a resource manager handle the likely case of the same asset being used by different sub-systems? Does each resource manager load it's own assets even if that means data duplication? Are all resource managers made global so you can access each one as you need it and if so, then why not just make one instance that can better manage ALL assets as a whole set?

     

    To me a resource manager (amongst other important systems) isn't the case of "there can be only one"... it's "there SHOULD BE only one!" as it's a system that is going to be accessed EVERYWHERE in the game where as a system like an entity manager is more than likely only going to be used in the gameplay section.... but even then why limit yourself right!? :)

    Share this comment


    Link to comment
    Share on other sites

     

    I'm curious.... why the anti-singleton sentiment here? It would make plenty of sense to use it for something like this in that one single, global instance is pretty much what you'd want from a resource manager!

     

    A single global instance? Fine. Make it a global. Or a global unique_ptr if you need to delay its construction until after main() is entered.

     

    But enforcing a single instance of it for no reason? Singletons are for putting a lock on an object preventing it from being constructed more than once.

    Singletons don't say, "I only need one.", singletons say, "Make absolutely sure that there can only ever be one."

     

    Singletons enforce that there is a 'single' instance. Some things are genuinely useful as globals: Like logging. But that doesn't mean you want to enforce that there is only one logging instance - you might want separate instances of loggers for separate subsystems (for example).

     

    std::cout and std::cin are both globals. Neither are singletons, which is good. What if you want a separate std::cout for error output instead of just general output? Well, create another global called std::cerr, and... oh wait, they already did. If std::cout was a singleton, std::cerr couldn't exist (unless they copy+pasted the code of that singleton, instead of just creating a second instance of it).

     

    The problem is not really truly with singletons (there are very few situations where they are useful, but those rare situations do exist), the problem is most people abuse singletons when they really just want a global (whether initialized before or after main() enters). A secondary semi-related issue is over-using (non-const) globals out of laziness or poor design (or genuine lack of time when being rushed on a project), but that wasn't being commented on.

     

     

    When used right, they are a very amazing and effective tool! Plenty of heavy commercial engines like Unreal or Ogre use them for a reason! I also agree they aren't always used right and very much abused! 

     

    I'd absolutely want only once instance of a render system or resource manager to be active at anyone time. I'd want to know that when I ask a resource manager for an asset that it's the same instance that I used when I loaded an asset into it earlier on.... hence the "enforcement" aspect of them. 

     

    In a single coder environment (as is the case for me now), I could see how using global pointers could be ok if you didn't want to go with singletons BUT when I worked for a large company with MANY coders, closely working away together on the same project, I can't tell you how many times singletons avoided so many problems where unprotected global instances caused them. Coders were (intentionally or not) duplicating instances of major systems and using the wrong instances of them. Coders are humans and humans aren't robots thus we will make mistakes! :)

    Share this comment


    Link to comment
    Share on other sites
    The only reason to "enforce" something is to prevent something bad to happen. And it is rarely the case for singletons. There may be several reasons to have more than one resource manager. You may for example want to separate assets by their lifespan or by type or by usage or by .. If you decide to use a singleton early in the development, you are limiting your ability to make modifications and extensions later on.

    A singleton is also often more difficult to implement, maintain and debug than most alternatives. You do not need to use a singleton to prevent copies of a global variable or to delay its initialization for example. unique_ptr s or private copy constructors or opaque interfaces are simpler and more flexible solutions to these problems.

    Finally, you are making everything a lot more verbose.

    Share this comment


    Link to comment
    Share on other sites

    Right! You are defensively coding to prevent bad things from happening and there are many tools at your disposal for such tasks... one of which is the singleton! 

     

    With respect to the case of multiple resource managers.... I ask again what I asked above:

    How would multiple instances of a resource manager handle the likely case of the same asset being used by different sub-systems? Does each resource manager load it's own assets even if that means data duplication? Are all resource managers made global so you can access each one as you need it and if so, then why not just make one instance that can better manage ALL assets as a whole set?

     

    In this case, you'd want a resource manager that does all the heavy lifting behind the scenes such as managing life span of assets based on types or usage. You'd also want that manager accessible from anywhere in the game so there is the case for a global variable. A singleton provides that along with extra defensive measures.

     

    Singletons are quite easy to implement. I use a templated version which works quite well and once implemented, you don't ever need to debug them again. Singletons basically came about as a way to consolidate all the various tricks you just mentioned in order to enforce a single instance of an object but in a cleaner way.

    Share this comment


    Link to comment
    Share on other sites



    Create an account or sign in to comment

    You need to be a member in order to leave a comment

    Create an account

    Sign up for a new account in our community. It's easy!

    Register a new account

    Sign in

    Already have an account? Sign in here.

    Sign In Now

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!