Sign in to follow this  

A note on resources and managers, why it is so hard to write a good one?

This topic is 3097 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

This topic is killing me, I want it to be simple as possible but not simpler, I can't just think of a way to do it. Ill start with what I came to: This is Reference counted interface, objects that wishes to be reference counted must inherit from here
        /** 
	 * Interface of reference counted object.
	 */
	class SKGE_API IReferenceCounted{
	private:
		uint m_RefCounter; /**< Number of references. */

	protected:
		/** 
		 * Add Reference.
		 */
		inline void addRef() { m_RefCounter++; }

		/** 
		 * Remove reference.
		 * @return New reference value after decrement.
		 */
		inline uint removeRef() { m_RefCounter--; return m_RefCounter; }

	public:
		/** 
		 * Constructor.
		 */
		IReferenceCounted():m_RefCounter(1){}
	};

Now I have Resource that implement the reference counter:
	/** 
	* Resource Handle.
	*/
	typedef ulong ResourceHandle;

	/** 
	 * Abstract Resource Class.
	 */
	class SKGE_API Resource: public IReferenceCounted{

		friend ResourceManager;

	protected:
		String m_Name; /**< Resource name. */
		ResourceHandle m_Handle; /**< Resource Handle. */
		bool m_Created; /**< Indicates if the resource was created. */
		ResourceManager* m_Creator; /**< Manager that created current resource. */

	public:
		/** 
		 * Constructor. Sets resource information, but does not load it.
		 * @param _name Resource name.
		 * @param _handle Resource handle.
		 * @param _creator Manager that created current resource.
		 */
		Resource(const String& _name, ResourceHandle _handle, ResourceManager* _creator):
		  m_Name(_name),
		  m_Creator(_creator),
		  m_Handle(_handle),
		  m_Created(false)
	    {}

	    /** 
	     * Destructor. Does not destroy the resource.
	     */
		virtual ~Resource(){}

		/** 
		 * Was the resource created?
		 * @return True if resource was created, otherwise false.
		 */
		inline bool created() const { return m_Created; }

		/** 
		 * Get resource name.
		 * @return Resource name.
		 */
		inline const String& getName() const { return m_Name; }

		/** 
		 * Get resource handle.
		 * @return Resource Handle.
		 */
		inline ResourceHandle getHandle() const { return m_Handle; }
	};

Note: Resource manager is a friend of Resource to be able to do reference counting on it (if you noticed all IReferenceCounted's methods are protected), I don't like friend classes/function but making addRef()/removeRef() public IMHO brakes the encapsulation so I had to choose between two bad methods. And the last one is ResourceManager
        /** 
	 * Abstract resource manager.
	 */
	class SKGE_API ResourceManager{
	private:
		static ResourceHandle sNextResourceHandle; /**< Next available resource handle. */

	protected:
		/** 
		 * Resource maps.
		 */
		//@{
		typedef stdext::hash_map<String, Resource*> ResourceNameMap;
		typedef stdext::hash_map<ResourceHandle, Resource*> ResourceHandleMap;
		//@}

		ResourceNameMap m_NameResourceMap; /**< Name->Resource Map. */
		ResourceHandleMap m_HandleResourceMap; /**< Handle->Resource Map. */
		String m_ResourcesDirectory; /**< Directory where resources sits. */

		/** 
		 * Resource removal.
		 * Totally destroy and remove resource from the system.
		 * @param _resource Resource to remove and destroy.
		 */
		void _removeResource(Resource* _resource);

	public:
		/** 
		 * Constructor.
		 */
		ResourceManager(){}

		/** 
		 * Destructor. Delete all resources
		 */
		virtual ~ResourceManager();

		/** 
		 * Get resources directory.
		 * @return Directory where resources sits.
		 */
		inline const String& getResourcesDirectory() const { return m_ResourcesDirectory; }

		/** 
		 * Set Resources Directory.
		 * @param _dir Resources Directory.
		 */
		inline void setResourcesDirectory(const String& _dir) { m_ResourcesDirectory = _dir; }

		/** 
		 * Get resource by name or handle.
		 * @return Resource or 0 if does not exists.
		 */
		//@{
		Resource* acquire(const String& _name) const;
		Resource* acquire(ResourceHandle _handle) const;
		//@}

		/** 
		 * Release resource and if needed delete it from system.
		 * @param _resoruce Resource to release.
		 */
		void release(Resource* _resource);
	};

Most of it self-explanatory, just to note acquire will search in the appropriate hash map and if found it will increase the ref calling addRef() and will return in, release on the other side will decrement the ref by calling removeRef() and if it reaches 0, it will call _removeResource() to totally delete and remove the resource from both hash maps. Till this point everything is fine. But most of you will notice that there is some obligations that if wont be followed will cause memory leaking. Assume the following:
//Till this point resource was loaded
{
 Resource* res = ResourceManager->acquire("Res1");
 //Do some stuff with resource

 //And here we must call this
 ResourceManager->release(res);
}

Ugly ain't it? Too complicated, as user of this system I don't care about the way of freeing resources, especially via resource manager (there will be more that 1, for textures, meshes, fonts, particles, etc). I could simply this by adding the following to Resource class:
void release(){
 m_Creator->release(this);
}

(Assuming the resource was loaded by the manager, in other cases I would say RTFM :] ) Now the following code could be replaced to this one:
//Till this point resource was loaded
{
 Resource* res = ResourceManager->acquire("Res1");
 //Do some stuff with resource

 //And here we must call this
 res->release();
}

Nicer right? But still not good, I still have to remember to free the resource. So I thought about Smart Pointers, something like:
//In resource manager change acquire to this:
 MageSharedPtr<Resource*> acquire(...);

//and the write like this
//Till this point resource was loaded
{
 MegaSharedPtr<Resource*> res = ResourceManager->acquire("Res1");
 //Do some stuff with resource

 //And we don't need to care about freeing it, cause the destructor
 //of MegaSharedPtr will call release() of the Resource*
}

And this is nice! But it brakes when I have to expand my resource to a more generalized resources like Texture/Mesh. Ill have to write conversions from MegaSharedPtr<Resource*> to MegaSharedPtr<Texture*> to be able to do it normally, cause Load() of Texture class would like to return Texture*, and acquire returns MegaSharedPtr<Resource*> that Ill have to cast to MegaSharedPtr<Texture*> or MegaSharedPtr<Mesh*> or etc.. So I'm kinda confused here and don't know how to do it right, it looks too complicated and I don't know how to simplify that :( I would like to have some suggestions or articles about the topic. Thanks a lot!

Share this post


Link to post
Share on other sites
Quote:
Original post by s.kwee
But it brakes when I have to expand my resource to a more generalized resources like Texture/Mesh.
Ill have to write conversions from MegaSharedPtr<Resource*> to MegaSharedPtr<Texture*> to be able to do it normally, cause Load() of Texture class would like to return Texture*, and acquire returns MegaSharedPtr<Resource*> that Ill have to cast to MegaSharedPtr<Texture*> or MegaSharedPtr<Mesh*> or etc..

So I'm kinda confused here and don't know how to do it right, it looks too complicated and I don't know how to simplify that :(
I would like to have some suggestions or articles about the topic.

Thanks a lot!


If you template your resource manager then your "get" function can do the type cast for you. Then you would only have to do something like Texture *texture = Manager<Texture>.Get("resource") or Mesh *mesh = Manager<Mesh>.Get("resource").

Share this post


Link to post
Share on other sites
Yes its nice, but since every resource is loaded differently (texture would get as parameter the file name, a boolean that says if there is a need to generate mip-maps, and probably a boolean to say if there is a need to clamp or repeat the texture, font on the other side will get file name and font size as parameters, particle will get the filename and time to live (for example)) so they all different and there is nothing common (except the get and release methods).

Share this post


Link to post
Share on other sites
1. Do not use your own reference counting. boost::shared_ptr is good. boost::weak_ptr is very useful with resource managers. boost::intrusive_ptr if you *really* want that. They're damned near standard too.

2. Assuming your resources will exist in a file is naive. Assuming they're all going to be in the same directory is just kind of criminally limiting your design. Not the biggest thing now, but...

3. Templated managers as stupid_programmer suggests are good, and will solve the root problem that you're asking about. It also allows you to inherit from the base manager and do specialization for sounds/textures/etc.

Share this post


Link to post
Share on other sites
Quote:
1. Do not use your own reference counting. boost::shared_ptr is good. boost::weak_ptr is very useful with resource managers. boost::intrusive_ptr if you *really* want that. They're damned near standard too.

I'm pretty anti-boost, I have my own conclusions about this library, and for now prefer not to use it.

Quote:
2. Assuming your resources will exist in a file is naive. Assuming they're all going to be in the same directory is just kind of criminally limiting your design. Not the biggest thing now, but...

As I move, new things come to my mind and I add them later, I can't think about everything now, since I also learn during the process. And I didn't say all resource is located in a file ;) For example in Texture Manager Ill of course provide two methods: createFromFril, createRaw. Also if you noticed about the variable m_ResourcesDirectory and methods to get/set it, so you kinda can change the folder :)

Quote:
3. Templated managers as stupid_programmer suggests are good, and will solve the root problem that you're asking about. It also allows you to inherit from the base manager and do specialization for sounds/textures/etc.

You mean something like:

template<class T>
class ResourceManager{
T* get();
void releat(T*);
};

class TextureManager: public ResourceManager<Texture>{
Texture* load();
....
};

class MeshManager: public ResourceManager<Mesh>{
Mesh* load();
....
};


Yes? I thought about this way, but C++ lacks of the inheritable templates like Java have, there for I can't limit the template to be a sub class of resource and could write something like:

class AbcManager: public ResourceManager<Abc>{
};


Where Abc is not a sub class of Resource, and by this logically I make resource manager from being a resource manager to anything manager. It's not bad I just dislike the style where you can't see lines between what you have and what you can have. However I will consider this as option because I'm really out of ideas.

Thanks again!

Share this post


Link to post
Share on other sites
As far as casting, this is what i'm rollin with..


/// The ResourceHandle wraps a shared_ptr to a Resource. The Resource is stored as a pointer
/// to the base class. The accessor function operator-&gt;() and Get() will cast the Resource
/// to the templated type of the ResourceHandle. Checks are made to ensure that the ResourceHandle
/// will only store a Resource of the templated type.
///
template &lt;typename T&gt;
class ResourceHandle
{
public:
ResourceHandle()
{ }

ResourceHandle(boost::shared_ptr&lt;Resource&gt; &resource)
{
if (!resource.get()) return;

Assert(resource-&gt;GetTypeID() == T::GetTypeIDStatic(), "ResourceHandle type mismatch.");
m_P = resource;
}

ResourceHandle(const ResourceHandle &rh)
{
operator = (rh.m_P);
}

void operator = (const ResourceHandle &rh)
{
operator = (rh.m_P);
}

void operator = (const boost::shared_ptr&lt;Resource&gt; &resource)
{
if (!resource.get()) return;

Assert(resource-&gt;GetTypeID() == T::GetTypeIDStatic(), "ResourceHandle type mismatch.");
m_P = resource;
}


T * operator-&gt;() { return static_cast&lt;T*&gt;(m_P.get()); }
const T *operator-&gt;() const { return static_cast&lt;T*&gt;(m_P.get()); }

T * Get() { return static_cast&lt;T*&gt;(m_P.get()); }
const T *Get() const { return static_cast&lt;T*&gt;(m_P.get()); }

bool IsSet() const { return m_P.get() != NULL; }
bool isStatus(ResourceStatus status) const { return m_P.get() ? m_P-&gt;Status() == status : false; }

private:
boost::shared_ptr&lt;Resource&gt; m_P;
};




As far as the different data required to construct a Resource. The Resource object has a Load function which takes a File pointer. This file could be a binary file, or an xml node. If it's not what the concrete resource expected then it complains, otherwise it just extracts the necessary data.

In other words, the depending information is injected into the Resource via an abstract interface File.

Share this post


Link to post
Share on other sites
bzroom
Thats nice, solves some problems! But do you have any kind of Resource Manager? How do you ensure that you have only single instance of particular resource in memory?

Share this post


Link to post
Share on other sites
Quote:
Original post by s.kwee
Quote:
1. Do not use your own reference counting. boost::shared_ptr is good. boost::weak_ptr is very useful with resource managers. boost::intrusive_ptr if you *really* want that. They're damned near standard too.

I'm pretty anti-boost, I have my own conclusions about this library, and for now prefer not to use it.


I submit that your own conclusions are wrong, at least regarding the shared pointer portion of the library. I've heard one mildly viable argument against that portion of the library, and I'm fairly certain it doesn't apply in your case. So what's the reasoning?


Quote:

Quote:
3. Templated managers as stupid_programmer suggests are good, and will solve the root problem that you're asking about. It also allows you to inherit from the base manager and do specialization for sounds/textures/etc.

You mean something like:
*** Source Snippet Removed ***
Yes? I thought about this way, but C++ lacks of the inheritable templates like Java have, there for I can't limit the template to be a sub class of resource and could write something like:
*** Source Snippet Removed ***
Where Abc is not a sub class of Resource


So what? What common behavior do all resources require? Will you ever need to store homogeneous collections of Resources together?

Quote:

It's not bad I just dislike the style where you can't see lines between what you have and what you can have.


There's no reason not to allow the things you're mentally trying to exclude.

Share this post


Link to post
Share on other sites
Every time I see a question like this I wish the article I'm writing for Gamedev on my resource manager were complete [grin]

Without going into too many specifics, I can point out a couple ares where you could improve your manager.

Firstly, "friend" is a huge red flag -- as you are aware -- that the design is flawed. In fact, "friend" is the absolute strongest relationship that can be expressed in C++, stronger even than inheritance, and as such, its power should only be called upon when absolutely necessary and justified. To require that any class which manipulates a refcount to be a friend of the ref-counted class is like using a sledgehammer to hang a picture-nail.

The ultimate issue you have is that your model of resources violates the single responsibility principle -- it is this poor separation of responsibilities that is leading to your problems. Your resource seems to "describe" the resource (in your case, naming it), hold its handle, hold its creation data (manager and created), and to presumably also act as the run-time data source for the resource (your example doesn't show this.)

In a well-separated system, such as mine (if I may be so bold), the separation of responsibilities breaks down something like this:

ResourceInfo - Identifies a resource within the system. This may be a filename, unique string, GUID, or other id.

ResourceData - Provides the data that the resource holds. The data (such as a texture or sound resource) doesn't and shouldn't know anything about how, or even whether it is "managed" -- it should be a class capable of standing on its own.

A Handle - The handle is what a client holds, and which allows access to the ResourceData instance associated with the handle. It is the handle which owns the ResourceData instance, and it is the handle which is ref-counted (multiple handles exist, not multiple copies of the resource). In my system, shared_ptr serves as the handle. shared_ptr is part of the tr1 extensions to the C++ standard library, and can be provided by boost if your platform does not yet support tr1, so this is a standards-friendly way of providing shared references to an object.

A Manager - The manager is what contains a distinct collection of resources. It provides methods for acquiring a resource handle (initiating loading if necessary), testing resource availability, and freeing a resource. Part of its job entails providing a map from ResourceInfo to its associated handle -- in my system, where handle is ref-counted, this means that the manager owns the initial handle (and hence, owns the ResourceData instance indirectly). This means that disposal of a resource can only come through a method call on the manager -- My system supports a "collect" method which removes a resource who's ref-count is 1 (which means that only the manager holds a reference). I can also force removal if I wish, but because handle is ref-counted, the ResourceData instance won't be destroyed until the last outstanding handle is deleted or goes out of scope, so there is no risk of deleting a resource that is still being used.

Loaders -- Loading resources form their persistent store is yet another separate responsibility. In my system, I specify the loader as a parameter to the manager method which acquires resources -- this is necessary since any call to acquire a resource may need to load that resource if it does not exist. If the requested resource does not yet exist, a new entry in the managers table (std::map) is created, and the loader is called through the supplied function pointer. This also means that, as long as there is enough memory, a resource will exist once this call is complete. To look for a resource that already exists (and without creating it if it does not) another member function, "find" is provided.


I'm very happy with my system and how it all fits together. It is completely generic (it can be applied to any Resource and ResourceInfo), separates loading into a separate code entity, makes no requirement on the Resource class (for example, resource classes are not required to support any interface, or derive from any base class) and is based entirely on standard library features -- which goes a long way to making it rock-solid and bug free. It even supports polymorphic resources -- that is, if the manager is templated against a base resource type, then it may manage any types derived from that base -- giving the client code tons of freedom.

Share this post


Link to post
Share on other sites
My resource management is very similar to Ravyne's in organization. I couple the Loader with the ResourceInfo, but it might be better if they weren't.

Share this post


Link to post
Share on other sites
Quote:
I'm pretty anti-boost, I have my own conclusions about this library, and for now prefer not to use it.
Are you also anti-standard-C++-library? If not, then you might reconsider your position on the Boost smart pointers, as the most commonly used among them (e.g. shared_ptr) have been incorporated directly into the most recent version of the C++ standard.

So why is it that you don't want to use shared_ptr and friends? Can you elaborate on that at all?

Share this post


Link to post
Share on other sites
Quote:
I submit that your own conclusions are wrong, at least regarding the shared pointer portion of the library. I've heard one mildly viable argument against that portion of the library, and I'm fairly certain it doesn't apply in your case. So what's the reasoning?

You would break any reason Ill give and I don't want this topic to become a holy-war of using or not using stl/boost. I just prefer to try my own, and break my mind in thinking about solutions before using a ready one.

Quote:
There's no reason not to allow the things you're mentally trying to exclude.

I agree, I just try to examine all possible solutions before choosing one.


Quote:
Every time I see a question like this I wish the article I'm writing for Gamedev on my resource manager were complete

Hehe I wish too, there is a little amount of articles about Resource managers.
Your idea of separating is very good and very true in opinion. I need just to enter more deeply in this and to see how I can implement something like this, so thanks for sharing :)

jyk
Sorry saw you message only after posting my own.
Quote:
Are you also anti-standard-C++-library? If not, then you might reconsider your position on the Boost smart pointers, as the most commonly used among them (e.g. shared_ptr) have been incorporated directly into the most recent version of the C++ standard.

No I'm not anti STL, and next time when you meaning the shared_ptr that is included in the standard library please say std/stl share_ptr and not boost. And I didn't know that shared_ptr was boosts, and I rarely used it.

Quote:
So why is it that you don't want to use shared_ptr and friends? Can you elaborate on that at all?

Some smart answer that will cause you to rethink about your position in favor boost I can't give. But if you insist Ill expand my answer to Telastyn. What I'm doing now is for my self and for my self education. I'm not payed for what I write and the chances Ill finish this are very low. However what I do get from this is knowledge. Have you ever asked your self why some courses teach you algorithms and data structures? Trying to recreate C++'s functionality via C? It would be easier to tell "you have std::list, this is a linked list, you can insert and remove items from it at any position via the functions insert and etc [don't remember them all right now], and if you want to sort it use std::sort, and for reverse use std::reverse". It would give the same effect, but when you implement your own you actually think "sort? how to sort? what is the best sort? how should I sort list?" "Reverse? how I can reverse list?". I do use things that I know how they work, I don't care writing it 20-30 times to get some reference counting functionality, what I do care is to understand why it works like this and when I know how it works, I would allow my self to use a ready one if its written better/is a standard.

[Edited by - s.kwee on June 23, 2009 3:41:39 PM]

Share this post


Link to post
Share on other sites
Quote:
Original post by s.kwee
You would break any reason Ill give and I don't want this topic to become a holy-war of using or not using stl/boost. I just prefer to try my own, and break my mind in thinking about solutions before using a ready one.


I'm not going to start an argument, and I can appreciate a good "learning exercise" as well as the next guy, but resource management is something which you presumably want to be practical -- it doesn't really make a good candidate for the "roll your own as a learning exercise" school of programming. In earlier versions of my manager, which was based on Scott Bilas' article in Game Programming Gems, it relied on a custom reference counting mechanism and shared pointer implementation, unnecessary friend'ing, and required that the manager be a singleton (instanced per resource type).

From my own experience, getting rid of the friend and singleton requirements got made the code infinately more flexible and nixing the ref-counting and shared pointer got rid of about 40% of the code and 2 or 3 classes. My Resource manager now, for all its features, is about 200 lines of clean, imminently re-usable code in a single .hpp file (its 479 lines with copious documentation and whitespace.)

Given, especially now, that it is part of the new standard, there is zero reason to avoid shared_ptr and its buddies, and it is near single-handedly responsible for the stability and elegance of my own solution.

Finally, boost is essentially a proving ground for future C++ features. The shared_ptr implementation adopted by the C++ standard is taken essentially verbatim from boost. Its basically like the ARB extensions to OpenGL, but for C++. Tons of incredibly smart people, such as those on or surrounding the C++ standards body, contribute to boost and its wise to benefit from their efforts and experience.

That's the last I'll say on it, but I would strongly encourage you to examine your bias against boost, and even the C++ standard library.

Share this post


Link to post
Share on other sites
To be able to use boost in a good way, one must be a good programmer, and I'm not.

Topic solved, kinda... Need time to rethink and replan my implementation.
Thanks to everyone who helped :)

Share this post


Link to post
Share on other sites
Quote:
Original post by s.kwee
Some smart answer that will cause you to rethink about your position in favor boost I can't give. But if you insist Ill expand my answer to Telastyn. What I'm doing now is for my self and for my self education. I'm not payed for what I write and the chances Ill finish this are very low. However what I do get from this is knowledge.

Programming is a broad subject. You seem to focus heavily on low-level aspects - so naturally, you're getting more familiar with those. However, the time you invest in that is time you did not invest on higher level aspects - interestingly, those are the things that you're getting stuck at now.

There's more to programming than low-level implementation details. Those things are useful, but don't forget about the big picture. You'd do well to also learn about good development practices.


/2 cents

Share this post


Link to post
Share on other sites
Quote:
Original post by s.kwee
To be able to use boost in a good way, one must be a good programmer, and I'm not.


If a library is well written, even shoddy programmers are able to use it in a good way. To be fair, most libraries are not well written. shared_ptr and weak_ptr though are pretty damned solid.

But I can't throw stones. I had my hand rolled linked list implementation that saw far more use than it should've while I was learning. Just remember that embracing library use is an important (and difficult!) lesson all on its own.

Share this post


Link to post
Share on other sites
Quote:
Original post by Telastyn
My resource management is very similar to Ravyne's in organization.

Mine too, although I break the load part further down to have codecs for encoding/decoding file formats and data sources that represent different protocols (like file, http, ftp etc).
However, the data sources are not part of the resource system but part of the virtual file system that is used by the resource system.

Share this post


Link to post
Share on other sites
About boost, after reading a bit about this library, I decided to give it a shoot again.

Lord_Evil
I see, but for now to me will be enough to load from file and create in memory. Thanks for the comment :)

Share this post


Link to post
Share on other sites
If there's one extremely powerful concept that I learned from working with component-based systems that would fit perfectly into a resource management system, it would be the distinction between type data and instance data.

One issue you tend to run into with traditional resource management is knowing when to clone a loaded resource versus simply reference the original. Not all resources fit into one category or the other, and I've found that by trying to pigeonhole the rest you end up in situations where you're either making tons of unnecessary clones or storing resource-related data outside of a resource object.

For example, a typical model resource might have a bone hierarchy and a set of animations. This would be "type" data that's exactly the same for every character that uses this model. However, each character would also need to have it's own animation state, which stores the current animation, frame, maybe a cached set of bone transforms, etc. This would be the "instance" data. You then have two classes, ModelResourceType which contains the first set of data and ModelResource which contains the second set of data. The resource manager loads exactly one instance of ModelResourceType from disc, memory, a database, wherever. The virtual filesystem handles that part. The client can then chose to create a ModelResource, which references the single instance of ModelResourceType, or just request the instance of ModelResourceType directly if that's all the information the client needs. What you end up with is a system that not only supports cloning and referencing, but also allows you to intelligently split your data into the appropriate categories. As an added plus, you end up caching the bare minimum amount of data necessary to describe and/or instantiate a resource.

It's not something I've experimented with in an actual resource management system, however I imagine it would work extremely well.

Share this post


Link to post
Share on other sites
Quote:
Original post by Zipster
If there's one extremely powerful concept that I learned from working with component-based systems that would fit perfectly into a resource management system, it would be the distinction between type data and instance data.
It's not something I've experimented with in an actual resource management system, however I imagine it would work extremely well.


this is pretty much how my (in development) resource system works. I have GameObjects (which are instances) and GameResources (which are type data). It works remarkably well.

The trick to using this layout, is to have the ResourceManager own the Resources, and the level, own the Instanced objects (as well as a list of resources it was responsible for loading). That way, the Level becomes the interface between the game and the resource manager :)

Also, i prefer to use templated factories, rather than template the ResourceManager. It means i can keep more things in one place (while internally having multiple factories for different things).

it is a lot of code, but hopefully will pay off with the flexibility that im looking for.

Share this post


Link to post
Share on other sites
Well after arguing and doing nothing else today, I started to recreate my Resource Management system.
I replaced the reference counting with boost::shared_ptr, however I can't figure out one thing.
Someone want to create a resource, he uses the appropriate resource manager (be it texture manager, or mesh manager), the resource is created inside the manager and have 1 reference to it, and right then it returned to the one who initiated the creation of it and this changes the reference to 2 (one reference held by the manager and another one by the initiator of the resource creation). During the runtime others would like to use the same resource, and resources reference will go up and down, until everyone who wanted to use the resource do not use it anymore, leaving it with count of 1 (only resource manager holds the reference now).
In this situation resource manager can not know that the resource is not shared anymore and can be freed. This is not the behavior I wish.
I probably mistaken here, but as far as I know no game load all its resources to memory, its loaded per level.
So I thought about loading needed stuff with level loading, and unload when level switched.

Anyone have any idea how I could implement this?
Thanks again.

Share this post


Link to post
Share on other sites
Quote:
Original post by s.kwee
You would break any reason Ill give


Yes, that's the point; so you can learn something. Stubbornness is not a good trait for programmers in general. (Not to be confused with persistence, which is often useful.)

Quote:
and I don't want this topic to become a holy-war of using or not using stl/boost.


A "holy war of using or not using stl/boost" would be sort of like a holy war over whether or not to brush your teeth.

And it's not "STL" any more. We're not talking about a third-party library. We're talking about the standard library of the language you are using. You can no more ignore std::string in C++ than you can ignore strlen() in C. (And FWIW, there are parts of the original STL that didn't make the cut for the C++ standard library. Though in most cases I wish it had.)

Quote:
No I'm not anti STL, and next time when you meaning the shared_ptr that is included in the standard library please say std/stl share_ptr and not boost. And I didn't know that shared_ptr was boosts, and I rarely used it.


jyk is talking about C++0x which is not commonly supported yet. The point is that it's coming, whether you like it or not.

Quote:
Have you ever asked your self why some courses teach you algorithms and data structures? Trying to recreate C++'s functionality via C?


Yes, and I still ask myself, after having gone through it and having heard all the arguments for doing so, many, many times.

Quote:
but when you implement your own


When?

Quote:
you actually think "sort? how to sort? what is the best sort? how should I sort list?" "Reverse? how I can reverse list?".


That's a good thing, though. If anything, people who do end up having to care about these things should be made to think about them, instead of being spoon-fed by a professor.

Quote:
what I do care is to understand why it works like this and when I know how it works,


And do you really feel that you have to implement it to understand it? What's wrong with reading a high-level description? Such as the one provided by the Boost documentation?

Share this post


Link to post
Share on other sites
Quote:
Original post by s.kwee
Well after arguing and doing nothing else today, I started to recreate my Resource Management system.
I replaced the reference counting with boost::shared_ptr, however I can't figure out one thing.
Someone want to create a resource, he uses the appropriate resource manager (be it texture manager, or mesh manager), the resource is created inside the manager and have 1 reference to it, and right then it returned to the one who initiated the creation of it and this changes the reference to 2 (one reference held by the manager and another one by the initiator of the resource creation). During the runtime others would like to use the same resource, and resources reference will go up and down, until everyone who wanted to use the resource do not use it anymore, leaving it with count of 1 (only resource manager holds the reference now).
In this situation resource manager can not know that the resource is not shared anymore and can be freed. This is not the behavior I wish.
I probably mistaken here, but as far as I know no game load all its resources to memory, its loaded per level.
So I thought about loading needed stuff with level loading, and unload when level switched.

Anyone have any idea how I could implement this?
Thanks again.
Is shared_ptr::unique() what you're looking for?

Share this post


Link to post
Share on other sites
Zahlman
Thank you for the comment, but I already understood my mistakes, and agree with what you said :)

jyk
Ok, but how I can notify the manager that the last resource released? The only way I see is calling X time units a function that will run on the resource manager managed resources hash table and will check for resources that are unique and will free them.

Share this post


Link to post
Share on other sites
I mentioned this briefly in one of my earlier posts.

What I have is a method called "collect" which walks through the table of resources and ejects resources who's reference count is just 1 -- I use the unique() method of the shared_ptr to determine if the reference count is 1.

Another version of collect() can optionally force ejection from the manager by setting its "force" parameter to true -- in which case the resource is ejected, regardless of its count. When this happens, the resource data is not actually destroyed until the last reference to it is deleted or goes out of scope. If the client requests to re-acquire that same resource again, a new resource instance is created, regardless of whether or not the old instance is still around, because there is just no way to know.

In general, you're going to want to call collect at known times (such as between frames or levels, when performance isn't critical), so I haven't done any work to specifically make my manager thread-safe, but I imagine it wouldn't be that difficult. With thread-safe support in place, one could spawn a new thread that would call collect periodically -- making the system more akin to active garbage collection.

Also, my resource system actually has an internal class ResourceMetaInfo, which wraps the ResourceInfo type, and which contains some additional information. Currently, this additional information amounts to a storage class specifier for resources. I have three storage classes for resources PERMANANT, TEMPORARY and MOMENTARY. PERMANANT resources aren't deleted unless "force" is set to true -- this is intended for system-wide resources (like menu graphics which must always be at the ready, for instance). TEMPORARY resources are not deleted when the refcount is 1 unless memory needs to be freed up (unless forced) and are intended for level-wide resources. MOMENTARY resources are those which are only likely to be needed for a frame or so, and are always deleted when the refcount is 1.

Hope that helps, feel free to ask any more questions you may have.

Share this post


Link to post
Share on other sites

This topic is 3097 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this