Jump to content

  • Log In with Google      Sign In   
  • Create Account

We're offering banner ads on our site from just $5!

1. Details HERE. 2. GDNet+ Subscriptions HERE. 3. Ad upload HERE.


Writing an Asset Manager


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
12 replies to this topic

#1 Vincent_M   Members   -  Reputation: 744

Like
0Likes
Like

Posted 07 January 2014 - 10:33 PM

I'd like to build an asset manager that can "own" allocated data loaded that's typically from files. That way, I can keep certain commonly-used assets that are persistent across multiple states of the game's life-cycle easily without having to write special-case code.

 

Plus, there's also no confusion of which pointer needs to release my asset from memory.

 

I'd like to somehow store all of my assets in STL maps where each different type of asset gets its own map. For example, if I add a texture to my AssetManager, a map that only holds pointers to allocated assets of type "Texture" will be created if it hasn't already. If I want to load up a SpriteCollection, I'd load and add it to the AssetManager. It'll generate another STL map to hold anything that's of type SpriteCollection if one doesn't already exist, or just add it to the one that does. Then, if I wanted to get a particular texture, I'd pass in the type and name (key) to that texture stored in the Texture's STL map. The getter method would be something along these lines:

Texture2D *texturePtr = AssetManager::GetAsset<Texture2D>("hero_character");

Another interesting thing is since I'm using multiple maps, I could have a SpriteSheet asset loaded that's also named "hero_character", and I'd be unique from the Texture2D pointer who's key is also "hero_character" because they're both different types of assets.

 

I'm new to templates, however... I'm trying to wrap my head around this, but haven't gotten it quite yet. Is this even possible in C++?



Sponsor:

#2 Hodgman   Moderators   -  Reputation: 31786

Like
4Likes
Like

Posted 07 January 2014 - 11:09 PM

Now, IMHO, the following code is a really bad idea, because the AssetManager is a singleton (and globals are bad, mmkay?)... but if you're ok with that, this code makes your example line of code compile:
#include <map>
#include <string>
struct Texture2D{};

template<class T> struct AssetLoader {};
//repeat this for each valid asset type to specify the loading function for it:
template<> struct AssetLoader<Texture2D>
{
	static Texture2D* Load(const std::string& name) { /*todo*/ return 0; }
};

//Creates a static std::map for each T that's used, and uses the above specialized loading functions to fill it
template<class T> struct AssetMapSingleton
{
	static T* GetAsset(const std::string& name)
	{
		auto i = assets.find(name);
		if( i != assets.end() )
			return i->second;
		T* asset = AssetLoader<T>::Load(name);
		assets.insert( std::make_pair(name, asset) );
		return asset;
	}
	static std::map<std::string, T*> assets;
//n.b. no cleanup code in here -- all assets will leak
};

//this is just here to convert from "foo::bar<T>" to "foo<T>::bar". If you change your example usage, then it's not needed ;)
struct AssetManager
{
	template<class T> static T* GetAsset(const std::string& name)
	{
		return AssetMapSingleton<T>::GetAsset(name);
	}
};

void test()
{
	Texture2D *texturePtr = AssetManager::GetAsset<Texture2D>("hero_character");
}
For a non-singleton method, you could re-write it as:
template<class T> struct AssetMap
{
	~AssetMap() { /*todo - for each asset in assets, free it*/ }
	T* GetAsset(const std::string& name)
	{
		auto i = assets.find(name);
		if( i != assets.end() )
			return i->second;
		T* asset = AssetLoader<T>::Load(name);
		assets.insert( std::make_pair(name, asset) );
		return asset;
	}
private:
	std::map<std::string, T*> assets;
};

void test()
{
	AssetMap<Texture2D> textures;
	Texture2D *texturePtr = textures.GetAsset("hero_character");
}

Edited by Hodgman, 07 January 2014 - 11:25 PM.


#3 ngoaho91   Members   -  Reputation: 253

Like
0Likes
Like

Posted 08 January 2014 - 01:15 AM

yes, it's possible, cocos2d-x already do it, you can take a look.

http://www.cocos2d-x.org/wiki/Texture_Cache



#4 Vincent_M   Members   -  Reputation: 744

Like
0Likes
Like

Posted 08 January 2014 - 01:18 AM

Very interesting! I thought about using a factory class, or at least, a generic factory method for loading my asset. This was mainly because I wanted loading to be static that would add it to the AssetManager, or an instance of an AssetManager if I wanted to treat them like packages (AssetManager instances) from repositories (zipped archives). I do agree that in some instances, singletons are bad. I think that for lower-level foundation work, however, they can be useful if they're actually meant to always stick around for entire duration of the software's life cycle.

 

Btw, I've noticed that some programmers generally make their method parameters const references (for example GetAsset(const std::string &name)). Is this just good practice to make parameters read-only that aren't meant to yield output?

 

It's a noob-ish question, but I've always wondered.



#5 Strewya   Members   -  Reputation: 1580

Like
1Likes
Like

Posted 08 January 2014 - 03:11 AM


Btw, I've noticed that some programmers generally make their method parameters const references (for example GetAsset(const std::string &name)). Is this just good practice to make parameters read-only that aren't meant to yield output?

 

It's a noob-ish question, but I've always wondered.

 

For non-POD types (such as std::string or your own structs/classes), using a reference in general is preferred because you avoid the copy when passing the parameter by value (which could be expensive/unneeded). The const is there because you, as a programmer, are making a guarantee that the original object, which you are referencing inside the called function, will not change it's value or, if a struct/class, it's internal state (value of any member variables).


devstropo.blogspot.com - Random stuff about my gamedev hobby


#6 NightCreature83   Crossbones+   -  Reputation: 3031

Like
3Likes
Like

Posted 08 January 2014 - 03:14 AM


Btw, I've noticed that some programmers generally make their method parameters const references (for example GetAsset(const std::string &name)). Is this just good practice to make parameters read-only that aren't meant to yield output?

Yes it specifies to the using programmer that this function will not mess with the parameters you pass into the function, by value semantics mean the same so the const is only needed for pointers and references.

 

The other benefit is that if you pass a big class through a reference is that it doesn't need to make a stack copy of that object to pass to the function, which makes the application faster. For example I had a bad copy in one of my asset loading functions, whilst what I wanted was an alias(reference), in debug asset loading was super slow untill I removed my copy of a std::vector.


Worked on titles: CMR:DiRT2, DiRT 3, DiRT: Showdown, GRID 2, Mad Max

#7 Vincent_M   Members   -  Reputation: 744

Like
0Likes
Like

Posted 08 January 2014 - 12:07 PM

 


Btw, I've noticed that some programmers generally make their method parameters const references (for example GetAsset(const std::string &name)). Is this just good practice to make parameters read-only that aren't meant to yield output?

 

It's a noob-ish question, but I've always wondered.

 

For non-POD types (such as std::string or your own structs/classes), using a reference in general is preferred because you avoid the copy when passing the parameter by value (which could be expensive/unneeded). The const is there because you, as a programmer, are making a guarantee that the original object, which you are referencing inside the called function, will not change it's value or, if a struct/class, it's internal state (value of any member variables).

 

Thank you! I came to this conclusion too, but wanted some one to verify. Since I'm re-working my foundation code before I start on my next project, I'll have to go through what I currently have, and make all of my input-only parameters constant references for all of my methods.

 


The other benefit is that if you pass a big class through a reference is that it doesn't need to make a stack copy of that object to pass to the function, which makes the application faster.

 

I have run into concerns about this on mobile projects where I'd pass in sub-classes of my base 'Object' class. These instances used to be anywhere from 1000 - 1600 bytes in those days. I've cut my memory footprint per object down to ~200 bytes for my new 'Transform' class, but that's still a little high. I'm also getting away from CPU-side graphics computation since mobile GPUs are picking up in speed and functionality (GPU skinning, eventually instancing via geometry shaders, etc).



#8 Vincent_M   Members   -  Reputation: 744

Like
0Likes
Like

Posted 08 January 2014 - 01:19 PM

Sorry for the double-post, but after looking over Hodgman's code, I had some questions. For example, you add a new element to your map by using insert()'ing a make_pair(). The way I usually do this is like so:

myMap[name] = childElement; // insert/overwrite assets[name] with a new asset pointer passed in

Depending on the case, I may check if an element of that key already exists, then return early if I don't want to overwrite. For example, when storing a map of pointers, I'll want to manually release allocated data on the heap first.



#9 Hodgman   Moderators   -  Reputation: 31786

Like
4Likes
Like

Posted 08 January 2014 - 09:43 PM

Sorry for the double-post, but after looking over Hodgman's code, I had some questions. For example, you add a new element to your map by using insert()'ing a make_pair(). The way I usually do this is like so:

myMap[name] = childElement; // insert/overwrite assets[name] with a new asset pointer passed in
Depending on the case, I may check if an element of that key already exists, then return early if I don't want to overwrite. For example, when storing a map of pointers, I'll want to manually release allocated data on the heap first.

 
Your code works, but it is ever-so-slightly wasteful, because it's equivalent to this:
T& reference = myMap[name];//if the key 'name' doesn't exist, insert a default value using this key
reference = childElement;//overwrite the existing value (or the just-created default value)
i.e. the [] operator does multiple tasks -- it checks if a key exists and inserts a default value if it doesn't exist, and then returns a reference to the value.
It's equivalent to:
myMap.insert( make_pair(key, T()) ).first->second
insert returns a pair<iterator,bool>, where the bool is false if the key already existed, and the iterator points to the key/value pair in the map.
Above, first is the iterator in this pair. insert(...).first->first is the key, insert(...).first->second is the value, and insert(...).second tells you whether the key existed or not.

It doesn't really matter in the end -- these two bits of code do the same thing (delete a pointer in a map and replace it with a new one) -- so just use whatever one you find to be easier to read ;)
Foo*& mapValue = myMap[key];
delete mapValue;
mapValue = value;
..................
auto i = myMap.find(key);
if( i == assets.end() )//key not present in map
	assets.insert( std::make_pair(key, value) );
else
{
	delete i->second;
	i->second = value;
}


#10 Vincent_M   Members   -  Reputation: 744

Like
0Likes
Like

Posted 08 January 2014 - 10:30 PM

Ok, that changes my perspective quite a bit. This is good because I'm still getting used to pointers. I use C# with Unity exclusively at work, so we don't get to bother so much about lower-level stuff like this. It's refreshing... lol

 

I'm leaning toward the singleton approach since it acts like what I want from a use-case perspective, but then there's really no way to automatically clear all of that memory, is that correct? I mean, if I were to have a static Release() method to release all resources for all maps, is there a way to grab each template type of the instanced map to free it?



#11 Hodgman   Moderators   -  Reputation: 31786

Like
4Likes
Like

Posted 08 January 2014 - 10:52 PM

Singletons are bad tongue.png  but here's one way you could clean up after them:
I split the AssetMapSingleton up, so that AssetMap<T> is a normal class, and then AssetMapSingleton<T> creates a global instance of that class.
AssetMap<T> inherits the IAssetMap interface, so that all AssetMaps can be iterated to have a Cleanup function called.

The AssetMap<T> constructor/destruction adds/removes itself from a list inside the global AssetManager -- this allows the asset manager to iterate through each asset map if it needs to. The AssetMap<T> destructor also does a last-minute Cleanup call, in case you forget to.

#include <map>
#include <string>
#include <vector>

class IAssetMap
{
public:
	virtual void Cleanup() = 0;
};
template<class T> struct AssetLoader {};

struct Texture2D
{
	void Release() { /*todo*/ }
};
template<> struct AssetLoader<Texture2D>
{
	static Texture2D* Load(const std::string& name) { /*todo*/ return 0; }
};


struct AssetManager
{
	template<class T> static T* GetAsset(const std::string& name)
	{
		return AssetMapSingleton<T>::instance.GetAsset(name);
	}
	static void CleanupAll()
	{
		if( maps )
		{
			for(auto i = maps->begin(), end=maps->end(); i!=end; ++i )
			{
				(*i)->Cleanup();
			}
		}
	}
	static void RegisterMap( IAssetMap* p )
	{
		if( !maps )
			maps = new std::vector<IAssetMap*>;
		maps->push_back(p);
	}
	static void UnregisterMap( IAssetMap* p )
	{
		assert( maps );
		maps->erase(std::find(maps->begin(), maps->end(), p));
		if( maps->empty() )
		{
			delete maps;
			maps = 0;
		}
	}
private:
	static std::vector<IAssetMap*>* maps;
};


template<class T> class AssetMap : public IAssetMap
{
public:
	 AssetMap() { AssetManager::RegisterMap(this); }
	~AssetMap() { AssetManager::UnregisterMap(this); Cleanup(); }//prevent leaks -- on program shutdown, do a last-minute cleanup
	void Cleanup()
	{
		for(auto i = assets.begin(), end=assets.end(); i!=end; ++i )
		{
			i->second->Release();
		}
		assets.clear();
	}
	T* GetAsset(const std::string& name)
	{
		auto i = assets.find(name);
		if( i != assets.end() )
			return i->second;
		T* asset = AssetLoader<T>::Load(name);
		assets.insert( std::make_pair(name, asset) );
		return asset;
	}
private:
	std::map<std::string, T*> assets;
};

//Creates a global AssetMap<T> for each T that's used
template<class T> struct AssetMapSingleton
{
	static AssetMap<T> instance;
};

void test()
{
	Texture2D *texturePtr = AssetManager::GetAsset<Texture2D>("hero_character");
}


#12 Vincent_M   Members   -  Reputation: 744

Like
0Likes
Like

Posted 28 January 2014 - 06:38 PM

I think I've finally got it! It's almost identical to your implementation, but I've learned quite a bit trying to get it to work. I needed the following classes in my implementation:

-IAssetList

-template<class T> AssetList

-template<class T> AssetLoader

-template<class T> AssetListSingleton

-Resources

 

The Resources class has been around for a while, and has used to hold the engine's default textures and shaders. I wanted to expand its use to allow programmers to load, release, flush and get resources. I provided template methods in Resources, which happened to be wrapper calls in to AssetListSingleton, and AssetLoader.

 

I wanted each type of asset to be in their own maps for a faster look-up. For example, Texture2D get its own map, Texture3D gets its own map, etc. AssetList exists as a template class to hold map of the template's type, and its constructor is also used to add itself to Resources. I wanted my Resources class to be the high-level manager that will store pointers to each of these lists in an STL vector for clean-up later on. I couldn't create a vector of type AssetList because it was a template. To get around this, I created an abstract class (interface) called IAssetList that the AssetList template could inherit from. Resources could hold a vector of IAssetList, and IAssetList's constructor would call a method in Resources, passing itself as a pointer, into Resources' AddAssetList() method to be added to the vector. Resource is also treated as a static class with a Shutdown() method that's called by the engine when the app terminates. Programmers can also use Flush() to dump all resources.

 

AssetLoader is a blank class that is specialized by classes that will be treated as assets. AssetListSingleton is a template class that's used to call its static methods to manipulate its static AssetList instance by Resources wrapper methods. As I understand it, different types of AssetListSingleton are generated whenever a different type of AssetListSingleton is referenced. This will also generate another static AssetMap instance, right? This list is accounted for thanks to its superclass' constructor.

 

By the way, are C++ templates generated at run-time when needed, or is it during compile-time?



#13 Irlan   Members   -  Reputation: 1770

Like
0Likes
Like

Posted 08 February 2014 - 08:12 PM

In my engine, the objects loads themself from a file stream. That was a personal design decision. I've tried to implement a ResourceTable<key, T*> with functors and creators, but ended up as a kind of unorganized code. If you want the post is here.

 

I'm not saying that you can't have a Asset/Resource Manager class to facilitate the "if-not-found, load". I'm saying that if you want a simple and flexible way of doing, the self loading can help.


Edited by irlanrobson, 08 February 2014 - 08:14 PM.





Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS