Sign in to follow this  
dietepiet

Game memory management

Recommended Posts

Hi all, Recent games often have such immense worlds that not all 3D models can be stored in memory at any single time. Therefore, I assume, 3D models are only loaded from disk when actually needed and some cashing mechanism is used to remove old models from memory when no longer needed. Now I was wondering, when should I remove older models? Should I wait until allocating new memory failed? It seems to me this would be just to late for smooth loading of new models. An alternative might be to keep track of the amount of bytes occupied by each model and by all models in total. This, however, is very hard to do (every allocated piece of memory actually takes up an extra few compiler specific bytes) or very inaccurate. Just unloading models when they are not used for some specific time is much simpler, but might require to much memory for one system and have unnecessary cash misses for systems with much memory. Anyone suggestions for a general solution to this problem? thnx, Dietger

Share this post


Link to post
Share on other sites
Quote:
Original post by dietepiet
Hi all,

Recent games often have such immense worlds that not all 3D models can be stored in memory at any single time. Therefore, I assume, 3D models are only loaded from disk when actually needed and some cashing mechanism is used to remove old models from memory when no longer needed.

Now I was wondering, when should I remove older models? Should I wait until allocating new memory failed? It seems to me this would be just to late for smooth loading of new models.
An alternative might be to keep track of the amount of bytes occupied by each model and by all models in total. This, however, is very hard to do (every allocated piece of memory actually takes up an extra few compiler specific bytes) or very inaccurate.
Just unloading models when they are not used for some specific time is much simpler, but might require to much memory for one system and have unnecessary cash misses for systems with much memory.

Anyone suggestions for a general solution to this problem?

thnx,
Dietger
Usually you load all reasources required for a level at load time. In cases where you can't do that (Huge / endless levels, or masses of content), then you usually allocate a certain space for resources - which may well just be "until there's no more space" for loading resources into video memory particularly. When there memory is full and you need another resource, you unload the least recently used resource from the cache, and load the new one.
You can decide to load the new model before you actually need it, for instance if you know there's a bit of the level coming up which has new enemies in it. It's also possible to unload resources from the cache when all instances are removed - for instance when you've killed all of enemy X in the level, there's no need to keep them around in memory.

Share this post


Link to post
Share on other sites
Thanx for your answer.

For video memory, your method works fine because there are only a few places in game you allocate video memory.
For system memory on the other hand, allocations (new operator in c++) happen all over the place. How can I easily check each such allocation for failure and call my model clean-up routine? especially when many such allocations happen somewhere deep in stl vector and list code.

thnx

Share this post


Link to post
Share on other sites
Quote:
Original post by dietepiet
Thanx for your answer.

For video memory, your method works fine because there are only a few places in game you allocate video memory.
For system memory on the other hand, allocations (new operator in c++) happen all over the place. How can I easily check each such allocation for failure and call my model clean-up routine? especially when many such allocations happen somewhere deep in stl vector and list code.

thnx
You won't be doing much, if any allocations on a per-frame basis, you'll only be allocating when you need to load new models into the cache. So long as you're careful not to leak memory or corrupt the heap, you're very unlikely to run out of memory, unless you're running on some fixed hardware like a console or other embedded system.
If you really want, you can just put a try {} catch block around your resource loading code to cache any allocation failures, and either exit the game or not display a model.

If you find you're doing a lot of allocations on a per-frame basis, you should seriously look into why you need to. vectors can be reserved, pools can be allocated ahead of time, and so on.

Share this post


Link to post
Share on other sites
Yeah, don't wait for an allocation to fail. Instead, have some sort of quota of loaded resources. If you go over the quota, free something first. Often you just free the one you used the longest time ago.

The quota could simply be just a maximum quantity of models/textures/whatever. If you want to try measuring the actual size of a resource and placing a limit on the total size used, it's usually easy to write a simple routine that returns an approximate value, and that's all you need really.

Share this post


Link to post
Share on other sites
Thanx Kylotan, i just now realized that allocating memory will succeed up until the point where the OS swap file reaches its maximum size. I obviously need to remove cashed models way before this point, preferable around the time the swap file is actually needed. so a more or less approximate resource limit will probably do just fine.

Share this post


Link to post
Share on other sites
Quote:
Original post by dietepiet
Thanx Kylotan, i just now realized that allocating memory will succeed up until the point where the OS swap file reaches its maximum size. I obviously need to remove cashed models way before this point, preferable around the time the swap file is actually needed. so a more or less approximate resource limit will probably do just fine.
Not quite - they'll continue until you run out of address space too. On a 32-bit OS, you've got about 1.5GB of space to play around in, since the upper 2GB is reserved for the OS, and all of the DLLs and the EXE itself could total up to around 500MB.
And of that 1.5GB, that's probably fragmented, not in one large chunk.

If you're on a 64-bit OS, running a 64-bit EXE then you're good for 8 TB of address space.

This is interesting reading if you'd like to read more about it

Share this post


Link to post
Share on other sites
Reaching that point is hard enough for a 32 bit application. You've only got 2 GB address space to yourself and part of that will be taken by things like mapped video memory, loaded DLLs and so on.

Also keep in mind that these numbers might change while your game runs. A common approach is to just choose an arbitrary limit, say, 1 GB, that represents what you expect from your target audience. That leaves 500 MB for mapped memory and 500 MB for normal use. Never underestimate the amount of memory hogging crap people install and keep running on their computers.

- You can use functions like GlobalMemoryStatus() and DirectX's GetAvailableTextureMem() to fine-tune this limit, but be careful. Otherwise, your game might run differently if the user starts Microsoft Office, then your game, then Alt+Tabs out and closes Office :)

Microsoft recommends: IDirect3DDevice9::GetAvailableTextureMem() returns the total available memory, including AGP. Allocating resources based on an assumption of how much video memory you have is not a great idea. For example, what if the card is running under a Unified Memory Architecture (UMA) or is able to compress the textures? There might be more space available than you might have thought. You should create resources and check for 'out of memory' errors, then scale back on the textures. For example, you could remove the top mip-levels of your textures.

- If you want to be smart, keep an LRU (least recently used) list of resources for quick unloading. There have been some articles and even a dedicated data structure here on gamedev.net.

Resources can be assigned a memory estimate, but it's also possible to shoot one's own foot with this. If the game prefers to unload large resources first, you might end up with your memory full of smaller resources and performance trouble where it would count - when larger resources need to be loaded.

- To avoid little hickups when loading resources during gameplay, you should begin to precache resources before they will actually be required. This way, you can set yourself a fixed time budget per frame that can be spend on precaching resources. If the time budget is up, stop loading resources and draw another frame.

Reading data from the hard drive doesn't eat up any CPU time (the DMA controller shovels data directly into RAM), preparing resources can easily be done in a thread, but creating the actual graphics resources will hog the system bus and, depending on what API and settings you use, might have to be done in the rendering thread.

Share this post


Link to post
Share on other sites
Thanx a lot, this sounds like really useful tips. Especially this one:
Quote:
- You can use functions like GlobalMemoryStatus() and DirectX's GetAvailableTextureMem() to fine-tune this limit

I did not know such functions existed. I will just inspect the memory usage each frame and start unloading if its to much relative to the available memory.
About loading, I already planned an asynchronous loading schema where objects are requested before they are needed, including some priority indicator. So a loading budget per frame fits right in.

Share this post


Link to post
Share on other sites
Trying to catch a failed allocation somewhere in code when you run out of memory is a bit kludgy. Instead, implement a resource manager of some kind where you request resources (models, textures, etc):

CMyModel* pModel = ResourceManager.LoadModel("MyModel.dat");

The resource manager's internal bookkeeping maintains a list of loaded models, how much memory they take up (don't fret too much about the couple of byte overhead on large datablocks for mem management), and a budget limit for models.

So, for example, lets say your limit sits around 4mb for models and textures. When the resource manager is asked to load a model, it can determine how much memory the model is going to require (you can store the total memory footprint at export time into the model's file header). If the manager determines that you'll be blowing your budget, at that point you'd dump the least-recently-used or whatever dealloc scheme you come up with.

The key here is to come up with a predefined budget for the different assets in your game. Come up ahead of time with a budget for textures, models, audio, etc etc. This is especially important when you're dealing with devices with very little memory (~256mb, 64mb, lower). If you're working on a platform with only, say, 64mb memory total and you allow resources to continue loading until your main memory is depleted, you'll run into a situation where models (or any other asset group) will take up the majority of your main memory, and you don't have any space for anything else.

Instead, with predefined budgets, you would say something like this:

Models : Max 16mb
Textures : Max 8mb
Audio : Max 10mb
Scripts : Max 4mb

etc... So if someone adds more than 16mb worth of models, your system can fail gracefully and it won't eat into Audio's slice of the pie. It doesn't matter if there's loads of main memory left, if a particular asset group exceeds its budget, it either needs to get trimmed down or individual assets need to get unloaded.

Finally .. there are a couple of good memory managers out there that'll tell you exactly how much memory your game uses, which allocations were made and when, and whether or not you had memory leaks. It's an invaluable tool to have.

Hope that helps,

Graf

Share this post


Link to post
Share on other sites
Thanx for your replay Grafalgar,

About budgeting different assets, I don't see the advantage over just loading anything needed until all memory is full, after which you unload assets based on some cashing policy. You can discriminate between assets in this policy if you like. What is the point of beforehand restricting the memory for certain assets?

Second, could you point me to any good open source memory manager? I have no idea about the quality and performance of different managers.

Share this post


Link to post
Share on other sites
Quote:
Original post by dietepiet
Thanx for your replay Grafalgar,

About budgeting different assets, I don't see the advantage over just loading anything needed until all memory is full, after which you unload assets based on some cashing policy. You can discriminate between assets in this policy if you like. What is the point of beforehand restricting the memory for certain assets?

Second, could you point me to any good open source memory manager? I have no idea about the quality and performance of different managers.


Ok, let's say you only have 100mb memory available on your system. If you fill it up with data until it's depleted you're effectively taking memory away from other systems, that may need it more than you.

So, let's suppose you have models and sounds files to load, and for argument's sake assume that that is all you need for your game, just to make the point easier.

With that in mind, you set up your system to load anything and everything until you run out of space. Also, you set it up so that models are loaded first, and then sounds.

In the beginning everything is fine - you have 20mbs of models, and 10mbs of sound. No problem. But then something interesting happens - all of sudden your model footprint spirals out of control and you find yourself loading with 100mb of model data, and no space available for audio.

So you decide, well, you'll load your least memory intensive systems first, and then your heavier ones. Ok, so you load 20mb of audio, 80mb left for models, and you start swapping/caching.

But then the process repeats and your audio spirals out of control - with each new sound file loaded, less memory is available to models..

In the end you try and become more and more clever in the order in which you load assets so that you can have a little bit of everything loaded into memory, without any one thing taking huge chunks of data. It ends up becoming a huge pain to manage, much more than its worth.

Now - to simplify it all, you give each asset group a budget .. you say that Models can never exceed 50mb, audio can never exceed 15mb, Scripts can never exceed 5mb, and so on (numbers made up). So when you load models, once you hit the magic 50mb limit, it starts swapping things out.

Never underestimate how greedy assets can become, especially when it comes to shared memory. Image two brothers sharing a room - eventually one takes over, leaving little to no space for other one.

As far as memory manager is concerned, look for Peter Dalton's memory manager. It's a good one. Also, don't worry about performance of the manager - just make sure to disable it in release mode.

Share this post


Link to post
Share on other sites
I second the note by Grafalgar. Budget each asset type. But even beyond that allocate memory differently, since each asset type has different usage patterns, and different definitions of quality-of-service.
Scripts HAVE to be loaded at the right times or the game doesn't run. But your 4096x4096 texture could be replaced with a 1024x1024 texture without breaking the entire game.

Set a fixed limit on particle memory, allocating from a pool instead of ever touching new/delete at runtime.
Set a fixed limit on models, but give priority to the lowest LOD of any model/texture first then the higher ones, so when you run out of memory you can still draw a low LOD versions untill room is freed up for the higher LODs again.
Set a fixed limit on sound, but split it between a pre-buffered cache for small sounds (under 5sec?) and a different set of buffers for streaming larger character conversations and music off the disk.
Etc.

Also think to break up the world into Instance and Global resource pools. The player model, sounds, textures, scripts, effects, shaders, etc. are ALWAYS going to be there make sure to make priority room for them first. Share sounds for things like walking on a surface. Share textures for scenery types. Don't mix too many instance types at once (make the player face off with groups of enemy type X,Y or A,B but not all 4 at once).

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this