Hi,

I am in the process of developing the resource management system for the tools part of my engine. I can currently load a model (.obj) using Assimp and create Meshes and Materials from it. I can also successfully load textures with STB_image. However there is no control over whether Meshes, Materials or Textures are created multiple times.

I have read most of the resource management posts on here and have come across the concept of the store, loader, cache and proxy division of responsibility. This is the design I would like to work off, in combination with resource handles.

I have gathered some notes so far but I'm still unclear on several points. For example:

1. Whether the ‘store’ is the conceptual replacement of the resource manager class and contains the load() function.
2. How the placeholder/proxy is swapped out with the real object?

Any clarification, links or pseudocode examples would be much appreciated. Are there any good sources of information or code examples of this approach?

This again is one of those not-a-single-solution questions so keep my input as a personal preference rather than a strict solution. We created our own archive data format to load asset libraries into memory by using memory mapped IO. This requires assets to be stored in chunks where one chunk fills an entire disk cache line and multiple OS pages. To speedup everything, our assets were not allowed to be split between chunks except if they are larger than 64k and so exceed a chunk. So we always have our entire data mapped to memory but OS manages loading of those which are currently accessed.

A problem that may occure is that assets are mixed in a way that keeps the OS have all of them in memory all the time, which leads to excessive page swapping. This is because you have to think about where an asset is currently needed. CPU based asset handling like level data is critical as the OS needs to grant access to the pages for the CPU to work properly while GPU based assets like textures/meshes are loaded only once into memory and then passed to the VRAM. So we also have to group assets together that are CPU based and those that are GPU based without have them cross chunk boundaries. This way we can access GPU assets only when passing them to VRAM (and probably perform some decompression) and have the OS swap their pages in and out less frequently. This however may be improved by also keeping unique assets together in the same location, that belong to one level or one mesh etc.

Btw. we make sure that assets are never cloned by identifying the content with a secure hash algorithm when creating the asset library.

Another problem on GPU bound assets you already mentioned is to use placeholders for as long as the asset is still in transition. We solved this using a resource handle, which is just a ref counted wrapper around a pointer/object ID in case of GPU related assets. In case of loading a GPU asset, the handle is returned immediately while a background task is started that pushes to resource to VRAM. The handle initially points to a small default assets which is loaded when the game starts and swapped at the moment when the background task finishes by an atomic operation. It may happen that a texture handle is pointing a cycle more or less to the default placeholder asset but that's ok for as long as it is loaded early enougth.

Asset unloading btw. is managed from the resource manager on certain criteria. Like in a database I worked on in my last job, data is accessed with a timestamp (or some kind of). If new data gets loaded, data that hasn't been touched since certain time is unloaded to keep memory usage as low as possible without spending more time than necessary on loading/unloading assets. Keep in mind that this is only used for Assets that occupy additional memory on the Graphics Card while CPU related assets are simply ref-counted pointer structs to the memory mapped data. And can be loaded/unloaded faster

@Shaarigan Thanks for your reply. This brought up several useful points.

I do plan to have an archive data format, something like .pak files. My loader classes (Texture/Mesh/etc) will have to deal with these through a virtual filesystem.

I hadn't considered having to separate CPU based and GPU based assets, that's something I'll bear in mind.

Do you pass the handle to your background task so that it can update the asset or do you use some form of callback function in the resource manager to update the handle?

I'm working on a reference counted unload system for the resource handles on the CPU side. Are the GPU assets you're referring to things like the VBO's and VAO's?

How is your resource manager made available to the rest of the system? I've currently separated mine out into handles, handles caches, Texture/Mesh/etc stores. That currently means I have to pass a number of stores into say the model parsing class.

MasterReDWinD said:
Do you pass the handle to your background task so that it can update the asset or do you use some form of callback function in the resource manager to update the handle?

We're using a kind of cooperative multithreading that is based on some small ASM instructions to create and swap entire callstacks on a CPU core. The handles are located in a special “pointer heap” and a pointer to their memory location is passed to the Task Runner. The task is then updating the handle using platform atomic operations or throws a warning/error to the Log System.

Everything in our game engine is considered to be a task, even rendering. This makes it dificult to debug but it has the benefit that tasks can rely on each other without locks. If a task finishes it signals the result and other tasks awaiting that signal can handle it. In theory, frame updates and rendering could also await a resource load but it is considered to be used only in few scenarios when we really need that Asset for example

MasterReDWinD said:
I'm working on a reference counted unload system for the resource handles on the CPU side. Are the GPU assets you're referring to things like the VBO's and VAO's?

VBOs, VAOs, Textures, Shader and everything else that occupies space in VRAM. I'm coming from an OGL background, where everything in there is just a plain 32bit integer, regardless if it is a Texture Handle, Shader Program Pointer or VBO address.

The most horrible bug I had once in acommercial game was that the game crashed for a memory overflow. We first thought it is some memory leak in our code but in the end it turned out to be a VRAM issue. Someone was creating Shaders over and over again every time when a game view switched, without releasing/reusing old shaders. That turned to an overflow in the VRAM after several hours of leting the game run. This was when I decided that in the engine some people and I are developing, we have to track everything that is supposed to occupy heap memory

MasterReDWinD said:
How is your resource manager made available to the rest of the system? I've currently separated mine out into handles, handles caches, Texture/Mesh/etc stores. That currently means I have to pass a number of stores into say the model parsing class

Once it came to the decision how we want to manage memory in our engine, I did some research, asked some people about best practice (so what we programmer usually do ? ) and the lead dev of one of my previous jobs suggested this blog post about memory management in BitSquid Engine. Starting from that day, everything in our game engine that is supposed to work on any kind of physical memory has to use an Allocator. It's small interface allows the most basic stuff like allocating memory returning a pointer, in case of classes/structs also initialize them by calling a constructor that matches some template parameter and releasing memory as well. It also tracks the number of allocations and the amount of memory requested and causes an assertion if the program leaves without fully releasing everything. This gives us an early clue about memory leaks.

That said, Asset management is based on the same principle as memory allocations. Everything that works on CPU/ GPU assets has to use an AssetAllocator which is inherited from the manager itself. Those allocator instances are usually created on start and we have a static singelton property to access basic Allocators for both, memory and Assets (which are used if programmers don't specify an allocator on class/struct creation). So to answer your question, our asset management is initialized once at start and made available as a generic AssetAllocator or as needed and passed explicitly to functions that work on Assets and any kind of memory

MasterReDWinD said:
store, loader, cache and proxy concepts… I have gathered some notes so far but I'm still unclear on several points. For example: 1. Whether the ‘store’ is the conceptual replacement of the resource manager class and contains the load() function. 2. How the placeholder/proxy is swapped out with the real object?

Those look like my words (store, loader, cache and proxy) from about a decade ago.

For both of your questions, you can see them in modern games all the time.

Have a look at the life cycle of game objects in Unreal Engine with this documentation from the engine's perspective, and in with Unity with this documentation from the script's perspective.

The store is the thing that owns them. It may be the game world, the simulation, a map, or some other container. Something somewhere controls what it means to have an object in the game, usually having it parented into a world hierarchy. You make a call like AddToWorld or SpawnActor or CreateInstance or Initialize. This can also work for individual components. It gives you back a game object or other which you can do something with. You as the user shouldn't know or care too much about its internal state, if it is displaying something or is loading asynchronously in the background or is at a particular LOD. This could be synchronous (blocking execution of the code until it is ready) or asynchrnous (returning flow control immediately while sub-components are processed) but that's an implementation detail.

The thing I called the proxy is what the script writer sees as the object. The script writer gets back a UE4 Actor or a Unity GameObject and they can use it immediately with no questions asked. From their perspective they received back a fully-formed, fully functional game object. You don't need to know, nor should you generally care, about what details the engine has actually loaded internally. All you need to know is that this object is fully usable for you.

For asynchronous engines the proxy object may have several function calls and events. For example, Unreal exposes PostLoad(), InitializeActorsForPlay(), PreInitializeComponents(), InitializeComponents(), PostInitializeComponents(), and eventually gets to BeginPlay() when proxy object has been fully realized in the game. Unity has fewer exposed elements since it is synchronous with the script, with Awake(), sceneLoaded events, and OnEnable(). But at the same time, you don't typically know details like which LOD or mipmap level is being used, nor do you care. That gets handled by the engine.

From the object script writer's perspective the loader and cache don't exist, or at least aren't something typically touched.

The loader may quietly switch out model LODs for a visual mesh or textures, or even substitute with a billboard, or even cull visibility entirely. When you create an object in the world you don't care where the loader is pulling resources from, if they're loaded from disk or cloned from another object or already held in memory. Similarly the script writer usually doesn't care about the cache. The cache may be quietly making every instance use a shared mesh model, or it might keep the model around in memory even after you've destroyed a game object. The cache might have existing shared copies of data structures, of audio clips, of physics models, and the script writer typically has no need to know about them.

With all that in mind, we can revisit your two items:

1. Whether the ‘store’ is the conceptual replacement of the resource manager class and contains the load() function.

That's a concept, not a specific function call. Something somewhere serves as a container. You may call it a SceneManager, a World, a Simulator, or some other name, but it is a “thing” which serves as the source for all the in-game “things” and controls their individual life cycles.

2. How the placeholder/proxy is swapped out with the real object?

From the scripting side, it doesn't matter. The script doesn't care if it is running at the highest level of detail or the lowest, it doesn't care how audio is being mixed, it doesn't care if the graphics texture is in a DDS format, PVR format, or S3TC format.

From the engine side, this all must happen transparently so it is whatever work the engine must do.

If the engine swaps out a mesh LOD it needs to identify which mesh is required, possibly generate a view-dependent mesh or view-independent mesh, possibly transfer the mesh to the video card, possibly morph between them so they don't pop, possibly unload the other mesh, and more.

The engine may switch over to use GPU instancing for some but not for others, changing the number of GPU draw calls without notifying the script.

The engine may use megatextures or sparse textures (so that only a portion of the texture is in memory at any given time) and this may be invisible to the game object script. That same task may require substantial effort on the engine, a sparse texture means knowing what data needs to be transferred to the graphics card and actually doing that work, even though it is never visible to the game object scripter.

The engine may also unload resources, such as dumping currently-unused textures or meshes or audio to save resource space while keeping the metadata around, and load them back up when they're eventually used. (This is more of an issue in game consoles then the PC, which can use virtual memory to accomplish much of the same.)

In each of these cases, the engine is doing work behind the scenes to make the best behavior, reduce the number of calls, share resources rather than duplicate them, etc., but that work is nearly always kept invisible to the programmers using the system. It is the engine's responsibility to quietly substitute out the concrete items, such as the LODs and the instanced objects, and the conceptual item used by the script writers (the GameObject or Actor or similar) remains available regardless of the actual data.

Shaarigan said:
Once it came to the decision how we want to manage memory in our engine, I did some research, asked some people about best practice (so what we programmer usually do ? ) and the lead dev of one of my previous jobs suggested this blog post about memory management in BitSquid Engine.

@shaarigan Many thanks for that link and your description of your AssetAllocator, I'll be sure to have a careful read of it. I have been following the book Game Engine Architecture and have managed to create provisional stack and pool allocators but knowing how to integrate them into the system as a whole will really help.

frob said:
Those look like my words (store, loader, cache and proxy) from about a decade ago.

@frob They are indeed. I have read over a lot of the resource manager related posts looking for patterns or ideas that stand out. Many thanks for providing clarification on these concepts and also for the links, which I will study.

frob said:
That's a concept, not a specific function call. Something somewhere serves as a container. You may call it a SceneManager, a World, a Simulator, or some other name, but it is a “thing” which serves as the source for all the in-game “things” and controls their individual life cycles.

I'm currently trying to employ handles in my system. A handle consists of two ints, an id and a validity check number. There is a handle cache class which stores a vector of assets, a vector of validity check numbers that goes with them, and a linked list of free spaces in the asset vector. I then have a texture store class with several functions that takes handle parameters and returns data about the texture e.g., width, height, OpenGL int reference. The store class also has a get_texture function which calls a load function in my texture_loader class. The texture_store also stores a map of string→handle for previously loads. The handle class and handle cache class are both parameterized. I don't have a proxy object yet, so for now the loader just returns the actual texture. I will have to make the loader return the proxy object and then run the real load asnychronously somehow. Hopefully that all makes sense.

So in summary my ‘cache’ is the handle cache as it holds the actual Texture Data. My store is the texture store as it allows other code to retrieve the Texture data, or initiate a load. My loader just loads the Texture, and the proxy will probably be a static white texture object (in the loader class). So I will end up with a store and loader class for each asset type. Have I applied the concepts correctly?