I totally agree my previous writers in what they wrote but have some kind of completition. For the purpose of going to multiple platforms I integrated a set of static functions that are inside the storage namespace handling some common type functions like creating/searching/moving/deleting files/folders and resolve special path requests for Application Path, Root Path and User Data where each path is equalized to use forward slashes everywhere (Windows doesnt has a problem with that) so query the application path may result in

C:/Program Files/myTest //Windows
/data/data/myTest.myCompany.com //Android

In the next step I set root directory (working directory in Windows) to something where my assets are and could then call

Storage::File::Open("./MyAsset", Storage::File::Open)

Finally anything in my environment uses streams instead of plain byte arrays where MemoryStream wrapps arround a byte array so they may be used too. Any stream is based on a general interface IDataReader/IDataWriter that supports Reading/Writing of a single byte, array of bytes and Position/Length properties where one specialization is StreamReader that wrapps arround another IDataReader for substream reading like in a package.

InFileStream ifs("./MyAsset", Storage::File::Open);
if(ifs)
{
   //DoStuff
   while(!ifs.Eof())
   ;
}
if(ifs)
  ifs.Close();

Is the standard way to open-read a file from disk in my case.

For production code it is totally ok to use direct file paths that come from an entry point file that you need. Otherwise your program wouldnt know what file to load and in the end where to start from. This entry point may be an asset package or an .ini file. That totally depends on your game environment.

This is a common technic used by for example The Elder Scrolls games to load the games content (Oblivion.bsa asset package as example) that may be replaced my modders to wipe change the game completely.

When you have got processed your asset entry point you need to give that to the asset manager for loading levels, main screen, whatever. I personally prefer asset hashes too as Hodgman already wrote because they have some advantages for protecting your assets (modders wouldnt now so fast what kind of file it is and what it is used for) and saving memory to use 4 bytes instead of a large indefined byte length string.

Again in production you could use strings or a file table hash <=> path conversion to get your assets loaded.

At least I would recommend to use some kind of internal managed asset storage (e.g. a package). I use an aligned, signed and encrypted custom format on this action but you may also use .zip for starting purposes. My format is build from 64 kb aligned chunks with an algorithm to minimize fragmentation to speedup asset reads from disk so the package handler contains a small filetable in its head that points into the chunk and offset. A stream finally needs to jump to position (chunk * 64kb + offset) and read N bytes equal to the file size.

Finally my asset reader functions are also static functions that take an instance of the abstract IDataReader (or sometimes also IDataWriter) interface putting out processed data like a texture from image or a mesh that may be handled however depending on the engine

Very nice answer ... thanks

Therefore, they will return a type that pretty much is ready-to-use. It feels weird to transform such to an e.g. byte-array and then cast it back to its original type. It feels like a weird overhead. I will need to do some benchmarks on how expensive this can become

I think Hodgman still wrote again anything to the conversion topic :wink:

Performance is something you shouldnt care about yet but if your environment is able to run multithreaded just stay away from the overlapped IO stuff and go into memory mapped files instead. This creates a memory page in the OSes holy private RAM and returns a pointer to that location depending on whatever type of access you like (read in this case would be enougth I think)

Taking that pointer and e.g. wrap it into a MemoryStream lets multiple threads access the same data at the same time.

I use a mix of memory mapping chunks from the asset package for something like scenes and buffered streams to the harddrive in a multithreaded task and event based fashion to speed things up. Needs a little synchronisation like look if a chunk was loaded into a mapped location that fits the position and size requested, otherwise look if it is a small asset and a stream traverse could fit it or else open a mapping for this and maybe the next chunk.

Thank you!

I have one more question about accessing the textures, as I have assets that are part of the main game, assets that have been created by a user (and encapsulated in their own custom/level_x/assets/texture file path) and assets that got added via updates.

Do you prioritise customised content, if a custom-level is being played? Because if the player decided to name one texture "wall.png" - coincidentally being a match to an actual texture of the main game - there is a chance of loading the texture from the main game instead of the intended one.

It is probably a better approach to give the derived LoadBlob type all possible texture-paths and maybe a priority-predicate? Then, custom-paths would have priority.

The only ugly moment is when both textures would be used.

This is how a file describing a game-object looks like at the moment:

custom_object_example =
{
texture = "wall.png"
-- other values
}

Are there any common solutions to this?

Prioritising custom content is a common solution.

Allowing custom configuration of asset loading priority is another solution. (e.g. An ordered list of directories or archives.)

I can imagine a system where a mod's config file contains a list of content sources, e.g. In my 'Kylomod' package I might have an assets.cfg file, like:

asset_paths = {
    "/": "kylomod/base_asset_overrides.zip",
    "/kylomod": "kylomod/data.zip"
}

The first line says "anything in kylomod/base_asset_overrides.zip is assumed to be located at the root of the asset directory, and would therefore override built-in data. The second line says "anything in kylomod/data.zip" is located at /kylomod, and so won't clash with anything else (probably).

This is working on the assumption that this mod has its paths added at a high priority to the asset loader system.

That's just one way of trying to accommodate both overrides and new data in a single system; I'm sure there are others.

Instead of explicit mount points, it might be better to use absolute paths directly in the packages and assume they're all mounted at the root. This allows user-defined content to unambiguously reference both its own assets and external assets (i.e. provided by other packages and/or the base game) without requiring any additional configuration.

Sounds simpler to me! But also rather prone to break. What if the level-name-changes? I doubt the player would rename all their classes. Seems to be a dependency that should not exist, solely from a standpoint of user-friendliness?

Changing asset names (which is essentially a remove/add as far as any versioning scheme is concerned) is prone to breakage regardless of the loading approach you use. It's often much better to just create a new asset and deprecate the old one, rather than remove anything. This is especially true if there's user content you don't necessarily control.

Not entirely sure what you mean. If the user decides to rename their level, I would have to iterate through all their asset-files and change all file-paths.

What do you mean by "create a new asset"? Seems weird to change or create new assets because of an altered level-name.

I probably misunderstood something, though.