Hi,

I'm currently working on replacing my resource manager that uses strings as identifiers to one that uses integers. I'm using a hash function to convert the file names to 32 bit integers at compile time. This is all works well but I have no idea how to continue from this point.

Since I have no copy of the actual filename at runtime, I can't use that to load the file from disk. One option would be to have a map that I can use to lookup the filename if I have to. Option two is passing the string alongside the id when I want to load the asset. Both seem a bit messy to me.

If you are using resource ids in your engine, how do you solve this issue?

In my engine/editor, each resource has a ResourceID object associated with it that contains information such as:

• Name - a human-readable string
• URL - the path to the file where the resource is located. This may be used if a resource is external (i.e. not embedded in the file). Can be absolute or relative path.
• Type - a const pointer to a statically-defined ResourceType, which is itself a name+UUID.
• 128-bit UUID which is used to identify/reference resource objects uniquely across all files. This is generated when a new resource is created (including when duplicating another resource), and then never changed.
• Lists of parent/child resources which are used to represent the ownership hierarchy.
• Flags - is the resource externally located (in which case use the URL to load it), is it locked for writing, is it hidden, is it copyable.
• Creation/modification date timestamps.

This information is stored in the header of my container format, which can store or reference any number of resources. When the format gets loaded, it can either read each resource data directly from the file (if embedded), or from the URL (if external flag is set). Usually big resources are external (e.g. audio files), while the rest are all embedded directly as blobs of binary data. The result of loading the container format is a ResourceSet, which contains resources of any type (through some template magic). Then, I can request resources from the ResourceSet either by name, UUID, or URL (UUID is preferred), by calling resourceSet→getResource<T>(uuid). Resources are wrapped in something similar to a shared pointer to control lifetime, but mostly I work with either raw pointers or “references” (which are a raw pointer to resource data + pointer to ResourceID). This works pretty well for my purposes, though building the type system and serialization in C++ was a chore and took many iterations.

So my suggestion is to use both integer ID + URL in some kind of ResourceID class type. Use the URL for loading the files and use the ID for other referencing where strings would be slow. You might also want to later add extra information like I have, and such a design will make it easier to extend.

Another solution could be to have a database, which will later also be used as a resource package/ archive file you ship with the game. When I worked for my previous employer for example, they wrote a game design software and so had need to store plenty of objects on disk. I created a database which uses memory mapped I/O and addressed the data in 64k chunks. Then we had different tables in 2k, 4k or 8k depending on how large the objects were we needed to store. The address of an object aka the object ID was then it's chunk + the table index + the offset inside the table.

We were able to address multiple terrabytes of data using a 5-byte object ID.

The transition from IDs used in the software vs. database IDs was performed using a B+ Tree which itself was stored in the database header. You can however use any other kind of index tree if you want.

You might have two different databases for production and release where one contains more information about an Asset than the other. I know we live in a storage paradise today but the more we have the more needs to be loaded from our games and how often do you need information like the type of an asset, filename or where it originaly was located; in a release build

Thank you for your suggestions! I will definitely think about these implementations at a later point but for now they both seem a bit overkill. For now what I want is a simple one line call to load a resource from the disk.

What do you think about this approach?

struct StringID {

public:
	constexpr explicit StringID(const char* str)
		: id(FNV1a(str)), str(str) {}

public:
	const uint64_t id;
	const char* const str;
};

I would pass this to the resource manager. If the resource is not in memory yet, use the string to load it.

Heisenberg33 said:
What do you think about this approach?

Whatever works for you is a good solution I guess but you should step back from hashing the name and instead generate a unique ID. Must not be a UUID but something similar could help. Its just (I don't think you'll get into that situation but) because hashing may result in clashes sometimes.

I'm using FNVb hashing and didn't had any clashes yet but consider your system to run for a while. We use for example this implementation to generate 64 bits and 32 bits unique IDs. It uses snowflake algorithm and is considered to have a very low collision ratio