Jump to content

  • Log In with Google      Sign In   
  • Create Account


#ActualHodgman

Posted 25 April 2013 - 08:04 AM

Every big commercial game engine will have it's own tools for this kind of thing, so when they choose to use a new library, they will want that library to provide a way to 'hook' all it's memory allocations.

e.g. if the library uses void* MyMalloc( size_t size ) { return malloc(size); } instead of malloc, then the engine authors can just change that one function (and the corresponding MyFree) to convert the library over to using the engine's memory system.

It's the same with a few other system features -- you'd also like to be able to hook any debug output (log/print statements) generated by the library, and if it makes use of worker threads, you'd like to be able to have it use your existing worker/job/task system rather than have it spawn it's own threads.

 

The features of Valgrind/memcheck are pretty important -- leaks, double deletes, invalid accesses, etc, so these features will certainly be present in the engine's memory tools. There'll probably also be a live reporting feature, when a log of memory events can be streamed out of the game over a socket/etc to a separate analysis program, so you can inspect the game's memory as it's running. This should show all current allocations (and maybe a history so you can step back in time to view past allocations) broken down by module (which will usually be tagged by programmers -- e.g. them saying that certain allocations go in the "physics" category). e.g. if you get a crash where something has accessed some memory that's already been free'ed, how useful would it be to be able to quickly rewind the memory history and see what objects have been allocated at that address most recently, and who allocated them?

 

When developing for consoles, you might just have, say, just 200MiB of RAM to play with and no virtual memory, so fragmentation becomes a very big concern.

e.g. After going in and out of your game/main menu 100 times, maybe you've got 100MiB of memory free, but it's so damn fragmented that the largest contiguous unallocated block is only 1MiB!

To debug these issues, you want to have a visual display of the address space so you can see where in memory each different module is allocating it's memory.

For performance analysis, you'll want per frame per module stats like number of allocations, time in malloc, size distributions, lifespan distributions, etc... For keeping memory usage low, you'll want to have reports on current, average and min/max memory usage per system. Ideally you'd be able to easily track how these numbers change over the weeks so you can get the heads up when someone implements a memory hogging new system.

If possible, it would also be able to traverse an ownership tree between allocations to help spot memory that isn't needed by the game but is still allocated anyway, like assets that are loaded by no longer required at this point in the game.

 

The memory tools in these engines will be quite complex, and most likely only available to and used by big console game developers.

Apart from the ones that come built into the big engines, the only stand-alone tool/middleware combo that I know of in this niche is Elephant/Goldfish.


#1Hodgman

Posted 25 April 2013 - 07:59 AM

Every big commercial game engine will have it's own tools for this kind of thing, so when they choose to use a new library, they will want that library to provide a way to 'hook' all it's memory allocations.

e.g. if the library uses void* MyMalloc( size_t size ) { return malloc(size); } instead of malloc, then the engine authors can just change that one function (and the corresponding MyFree) to convert the library over to using the engine's memory system.

It's the same with a few other system features -- you'd also like to be able to hook any debug output (log/print statements) generated by the library, and if it makes use of worker threads, you'd like to be able to have it use your existing worker/job/task system rather than have it spawn it's own threads.

 

The features of Valgrind/memcheck are pretty important -- leaks, double deletes, invalid accesses, etc, so these features will certainly be present in the engine's memory tools. There'll probably also be a live reporting feature, when a log of memory events can be streamed out of the game over a socket/etc to a separate analysis program, so you can inspect the game's memory as it's running. This should show all current allocations (and maybe a history so you can step back in time to view past allocations) broken down by module (which will usually be tagged by programmers -- e.g. them saying that certain allocations go in the "physics" category).

When developing for consoles, you might just have, say 200MiB of RAM to play with and no virtual memory, so fragmentation becomes important. After going in and out of your game/main menu 100 times, maybe you've got 100MiB of memory free, but fragmented so that the largest contiguous unallocated block is only 1MiB... To debug these issues, you want to have a visual display of the address space so you can see where in memory each different module is allocating it's memory.

For performance you'll want per frame per module stats like number of allocations, time in malloc, size distributions, lifespan distributions, etc...

If possible, it would also be able to traverse an ownership tree between allocations to help spot memory that isn't needed by the game but is still allocated anyway, like assets that are loaded by no longer required at this point in the game.

 

The memory tools in these engines will be quite complex, and most likely only available to and used by big console game developers.

Apart from the ones that come built into the big engines, the only stand-alone tool/middleware combo that I know of in this niche is Elephant/Goldfish.


PARTNERS