Redirected from: http://gamedev.stackexchange.com/questions/120132/should-you-load-assets-resources-at-runtime-or-compiletime

Basically, I am creating a advanced game, and am currently implementing the Asset Manager. With my setup it is extremely trivial to embed data into the executable (It's literally one line per file). Of course, embeding into the executable is faster than reading from disk. However, I see many big games use the later anyway. Why is this? Which approach should I aim for? Why? Can you give me any (dis)advantages for using one or the other?

It's a trade-off. You should probably do both, depending on the purpose of the asset.

"Compile-time" loading, that is, embedding the resource into the executable, has the advantage of the resource always being available. There's no (reasonable) way for the asset to "not be there," so it's a good thing to use for very important, low-level assets, such as your default shaders, fallback "object not found" models and materials, et cetera. However, every resource you embed this way bloats the size of your executable, and they're hard to change or update, so you should use them sparingly and only for things that are very important.

"Runtime loading" is reading the assets from a file on the disk or elsewhere. These are way easier to update and patch and you can usually control the layout of the file better (or at least with less effort), which is useful for optimizing loading and patching. The downside is that the data can go missing relatively easily, so you have to have fallbacks or error handling in place.

Most of your data should probably be loaded at runtime. The advantages in control (and thus in perf) are significant. It's not true that "compile time loaded" data is faster at all. That's basically never the case in practice, especially if you're going to put all your data into the executable like that. It still gets read from the disk and if there's enough of it will get paged in and out just like anything else. Embed only the most critical of resources.

Resource loading has two phases: first you acquire a chunk of bytes, and then you parse or interpret them. It's pretty easy to support both runtime and compile-time assets, you just abstract away two different ways of getting that chunk of bytes (from a file, or from a well-known pointer to a chunk of bytes in your data segment). The rest of the resource handling is independent of the process by which the btyes were obtained.

"Compile-time" loading, that is, embedding the resource into the executable, has the advantage of the resource always being available. There's no (reasonable) way for the asset to "not be there," so it's a good thing to use for very important, low-level assets, such as your default shaders, fallback "object not found" models and materials, et cetera. However, every resource you embed this way bloats the size of your executable, and they're hard to change or update, so you should use them sparingly and only for things that are very important.

"Runtime loading" is reading the assets from a file on the disk or elsewhere. These are way easier to update and patch and you can usually control the layout of the file better (or at least with less effort), which is useful for optimizing loading and patching. The downside is that the data can go missing relatively easily, so you have to have fallbacks or error handling in place.

Most of your data should probably be loaded at runtime. The advantages in control (and thus in perf) are significant. It's not true that "compile time loaded" data is faster at all. That's basically never the case in practice, especially if you're going to put all your data into the executable like that. It still gets read from the disk and if there's enough of it will get paged in and out just like anything else. Embed only the most critical of resources.

If you wanted to go nuts with building assets into executables instead of reading files, you could put them in DLLs. Then you can load and unload the assets and the code that goes along with them at run time. Each level could have its own DLL. However, I must say, this is probably a bad idea.

I can swear I've worked on some project where they had a DLL filled with nothing but assets, but I don't remember what it was.

I can swear I've worked on some project where they had a DLL filled with nothing but assets, but I don't remember what it was.

Go back about 20-25 years, quite a few windows games (such as they were) would do this quite often.

Starting with Windows 3.0's introduction of virtual memory meant far less hassle for developers. No more futzing about with tools like VROOM or extended memory management swapping your application in and out.

It meant your application could easily exceed 640KB without using overlays, and you could dump all you wanted into a giant DLL with one or two megabytes of data -- enormous at the time -- and not have to worry much about memory management. Magic happened.

While it may not seem like much today, imagine how many thousand 8-bit color sprites you can fit into that. The magic of virtual memory simplified things for the programmers, and although some things were slower development was faster and cheaper, and therefore it was often leveraged for cost.

Memory management is necessary when your assets are all custom data loaded from files. But if all your assets can be put into OS-friendly objects that can be referenced, loaded, unloaded, and otherwise handled by the OS when you only use a resource ID, it is a hard argument to not to use it when it fits your game, and even when it doesn't exactly fit your game.

If you wanted to go nuts with building assets into executables instead of reading files, you could put them in DLLs. Then you can load and unload the assets and the code that goes along with them at run time. Each level could have its own DLL. However, I must say, this is probably a bad idea.

Putting stuff -- assets and code -- into DLLs for hot-reload purposes is still a pretty valid technique. Unreal supports it (for code, but you could force it into working for assets). It can be rather a pain to get done right, though, and if you're talking about doing it for assets only it's probably way more work in the long run than just implementing a more vanilla approach for hot-reloaded assets from files on disk.

FYI: I use C (fopen for runtime loading, and objcopy/xxd for compile time loading)

This is what I have:

#define NUM_TEXTURES 1
#define LoadTextures(LoadTexture)\
LoadTexture(TEXTURE1, texture1, "/assets/textures/texture1.png")\
/* ... */

#define LoadTexture(a, b, c) a,
typedef enum texture_id {
LoadTextures(LoadTexture)
} TextureID;
#undef LoadTexture

textures[NUM_TEXTURES];

int main(void) {
#define LoadTexture(a, b, c) textures[a] = LoadTexture(c)
    LoadTextures(LoadTexture)
#undef LoadTexture
    return 0;
}