Hi ,all

Our game has some problems with memory, there are two many third party library in our game . most of them are closed source, I have some trouble with profilering it . I am newbie for this ,I hope I can have some insight here.

Situation:

After running a long time , allocing memory will be failed.

Analyzing:

Reason 1: There may be some memory leak when game running.

Reason 2: After a long time running , there may be memory fragmentation in the memory space!

Question:

For reason 1, I tried to hook heapalloc and heapfree , then output some log for analyzing , but if I hooked these api ,the game run very slowly, which will lead to net error happenning , the test failed; I also try umdh in windows debugging tools, it has help me find some leak.

Is there any other way help me to find a leak in the third party library???

For reason 2, I use vmmap to see the unused space. Is there some other way to find the code which caused these memeory fragment?

Is there any suggestions for memory manangement with so many third party library in a game?

For reason 1, I tried to hook heapalloc and heapfree , then output some log for analyzing , but if I hooked these api ,the game run very slowly

If hooking allocation functions is causing the game to run that slowly, that's probably an indication that you're doing _waaaaay_ too much allocation during the game loop.

Which in turn probably causes fragmentation.

Also means you've got that much more room to mess up and leak (though with modern C++ programming, that should be rare). It also could mean that fragmentation is leading to allocation failure.

An idealized game preallocates as much memory as it needs before entering a level or whatnot and never allocates again. Larger open world games can get by with good object pools and paging techniques, though this takes a lot of effort and discipline.

If your third party libraries are the source of allocations, you have some mitigations. Most third-party libraries made for games allow hooking the library's allocation routines. You could use those hooks to pass the libraries straight to the fast allocator and then only override the allocator for your game code, and verify whether your own code is leaking. Alternatively, you can enable the leak detector individually for each library to track down the culprit. If it's just fragmentation, hooking the offending library to use its own small block pool allocator may be the solution.

I head that valgrind is quite popular, I have not used it myself but I friends that does and praise it.
If you want to try it out you can find it here:

http://valgrind.org/

As Sean said, allocate as little as possible. Figure out you need 50,000 particles at once, but never more. Then preallocate 51,000 or some safety buffer. Then use asserts if you ever breach the 50,000 mark. Recycle already allocated memory with memory pools, etc.

There are also free tools out there written by gamedevs like https://mtuner.net/

I head that valgrind is quite popular, I have not used it myself but I friends that does and praise it.
If you want to try it out you can find it here:

http://valgrind.org/

Valgrind is a poor fit for games, unfortunately. Its biggest problem is that it effectively runs as a full emulator and will make a game run 100x slower; if just hooking allocations is too slow, that's certainly not possible. Valgrind also isn't compatible with Windows which is a deal breaker for most game developers.

There are toolkits that do similar things which manage to be both far faster and Windows-compatible, though perhaps without quite as much accuracy.

There are also free tools out there written by gamedevs like https://mtuner.net/

Highly recommended. One of our engineers just hooked this up to our (big AAA) game recently and we've been impressed. A custom tool like Hodgman recommended can still have a lot of benefits that you won't get with MTuner but a custom tool would probably be overkill for the OP.

Someone needs to write one of these as open source. It's an easy enough problem to tackle for someone with the time and experience. :)

There are also free tools out there written by gamedevs like https://mtuner.net/

Thanks ,I will try it latter