Jump to content
  • Advertisement
Sign in to follow this  
YoYoYonnY

Should you load assets/resources at runtime or compiletime?

This topic is 815 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Redirected from: http://gamedev.stackexchange.com/questions/120132/should-you-load-assets-resources-at-runtime-or-compiletime

 

Basically, I am creating a advanced game, and am currently implementing the Asset Manager. With my setup it is extremely trivial to embed data into the executable (It's literally one line per file). Of course, embeding into the executable is faster than reading from disk. However, I see many big games use the later anyway. Why is this? Which approach should I aim for? Why? Can you give me any (dis)advantages for using one or the other?

Share this post


Link to post
Share on other sites
Advertisement

"Runtime loading" also lends itself better to reloading files with the application running, which can allow a much faster iteration time.

 

If you have a system set up for e.g. reloading files if they are changed/on user action, you can use this to tweak various values without having to recompile the entire application every time there is a change.

"Hm, the movement speed of that enemy is a bit high" -- adjust, save, reload enemy settings without restarting game and progressing to that point again.

Share this post


Link to post
Share on other sites

I don't see how "compile time loading" could be any faster, you might be confusing this with the fact that you're paying the cost of loading the resource when the program is loaded, rather than paying it when you call LoadAsset(...) or somesuch to load the resource from a file at runtime. In other words, its not faster, you've just failed to measure the compile time scenario at all.

 

Josh gave a nice overview of pros/cons, and suitable use-cases. You definitely don't want to "compile time load" all your assets, least of all on the false premise that its somehow faster. Particularly, the down-side is that whatever is in the data segment is in memory whenever your program is -- that means if you have 4GB of assets, they're all in memory even if you're only using 400MB of them in the current level or scene, and your minimum requirements will reflect that. Now, with virtual memory that's not the whole story and the OS will jump through hoops making your game work, but--and here's the point--had you loaded those assets at runtime then you only will have in memory exactly what you need in a given level or scene; and what's more, you gain the flexibility of loading lower-fidelity versions (e.g. smaller mip-levels) of assets if you needed to get your memory footprint down even smaller, or the reverse to load higher-fidelity versions for users with tons of memory.

 

The great majority of your assets should be loaded at runtime. Personally, I would only consider compile-time-loading assets which, if missing, would mean that the engine, not the gamewould be unable to continuing to function as designed. Even at that, I would strongly consider loading them at run-time, as soon as possible, rather than embedding them in the executable just because it still affords greater flexibility.

Share this post


Link to post
Share on other sites

"Compile-time" loading, that is, embedding the resource into the executable, has the advantage of the resource always being available. There's no (reasonable) way for the asset to "not be there," so it's a good thing to use for very important, low-level assets, such as your default shaders, fallback "object not found" models and materials, et cetera. However, every resource you embed this way bloats the size of your executable, and they're hard to change or update, so you should use them sparingly and only for things that are very important.

 

"Runtime loading" is reading the assets from a file on the disk or elsewhere. These are way easier to update and patch and you can usually control the layout of the file better (or at least with less effort), which is useful for optimizing loading and patching. The downside is that the data can go missing relatively easily, so you have to have fallbacks or error handling in place.

 

Most of your data should probably be loaded at runtime. The advantages in control (and thus in perf) are significant. It's not true that "compile time loaded" data is faster at all. That's basically never the case in practice, especially if you're going to put all your data into the executable like that. It still gets read from the disk and if there's enough of it will get paged in and out just like anything else. Embed only the most critical of resources.

If you wanted to go nuts with building assets into executables instead of reading files, you could put them in DLLs. Then you can load and unload the assets and the code that goes along with them at run time. Each level could have its own DLL. However, I must say, this is probably a bad idea.

I can swear I've worked on some project where they had a DLL filled with nothing but assets, but I don't remember what it was.

Share this post


Link to post
Share on other sites

I can swear I've worked on some project where they had a DLL filled with nothing but assets, but I don't remember what it was.


Go back about 20-25 years, quite a few windows games (such as they were) would do this quite often.

 

Starting with Windows 3.0's introduction of virtual memory meant far less hassle for developers.  No more futzing about with tools like VROOM or extended memory management swapping your application in and out.  

 

It meant your application could easily exceed 640KB without using overlays, and you could dump all you wanted into a giant DLL with one or two megabytes of data -- enormous at the time -- and not have to worry much about memory management.  Magic happened.

 

While it may not seem like much today, imagine how many thousand 8-bit color sprites you can fit into that. The magic of virtual memory simplified things for the programmers, and although some things were slower development was faster and cheaper, and therefore it was often leveraged for cost.

 

Memory management is necessary when your assets are all custom data loaded from files. But if all your assets can be put into OS-friendly objects that can be referenced, loaded, unloaded, and otherwise handled by the OS when you only use a resource ID, it is a hard argument to not to use it when it fits your game, and even when it doesn't exactly fit your game.

Share this post


Link to post
Share on other sites
If you wanted to go nuts with building assets into executables instead of reading files, you could put them in DLLs. Then you can load and unload the assets and the code that goes along with them at run time. Each level could have its own DLL. However, I must say, this is probably a bad idea.

 

 

Putting stuff -- assets and code -- into DLLs for hot-reload purposes is still a pretty valid technique. Unreal supports it (for code, but you could force it into working for assets). It can be rather a pain to get done right, though, and if you're talking about doing it for assets only it's probably way more work in the long run than just implementing a more vanilla approach for hot-reloaded assets from files on disk.

Edited by Josh Petrie

Share this post


Link to post
Share on other sites

FYI: I use C (fopen for runtime loading, and objcopy/xxd for compile time loading)

 

This is what I have:

#define NUM_TEXTURES 1
#define LoadTextures(LoadTexture)\
LoadTexture(TEXTURE1, texture1, "/assets/textures/texture1.png")\
/* ... */

#define LoadTexture(a, b, c) a,
typedef enum texture_id {
LoadTextures(LoadTexture)
} TextureID;
#undef LoadTexture

textures[NUM_TEXTURES];

int main(void) {
#define LoadTexture(a, b, c) textures[a] = LoadTexture(c)
    LoadTextures(LoadTexture)
#undef LoadTexture
    return 0;
}
Edited by YoYoYonnY

Share this post


Link to post
Share on other sites
I'm not sure what you're trying to convey there? Is that supposed to be an explanation of why (you think) this is faster? Or is it simply informational/posted for further critique or advice? If the former, it doesn't address the above points regarding the performance of embedded resources.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!