• Advertisement
Sign in to follow this  

Analysis of memory allocations

This topic is 4149 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi... I have a memory problem with my engine and i can't find what is going wrong. Based on Windows' task manager my application consumes 140MB of memory, and the problem is that there is no apparent reason for that. Let me explain. When the loading phase finishes, the task manager reports about 22MB which seems logical. Until then the only thing that i can imagine taking the most of the allocated memory is model geometry, because this is the only thing that is loaded and kept for later use. Other things include image filenames, material properties, etc. After that, when the first frame is going to be rendered, i create all vertex buffers required by the visible portion of the scene, as well as load all the textures for the visible part. Because of that, i first tried to see if there is a memory leak in the image loading part but there isn't any. What happens is when an image is uploaded as a texture to GL, its data are freed from system memory (and i can see the ups and downs of the total consumed memory from task manager when a big texture is loaded). Except from images/textures the only other thing that happens at run-time is vertex/index buffer creation from the loaded geometry data. This seems to be the problem, but i can't understand why. GLExpert reports that all vertex/index buffers are allocated in video memory. And even if task manager reported video memory in the application's memory footprint, i wouldn't expect it to be more that 50MB (double the memory footprint from the loading phase). To the question. Is there any program/library which can tell me where and when all the allocations happen, and the true allocated memory for each one of them? Is there any other way i can get this info without writing custom memory allocators on my own? I will do eventually (if nothing else helps), but i would like to try something simpler/faster first. Notice that this is about unmanaged code (native C/C++). Thanks in advance. HellRaiZer PS. Sorry if this has been asked again, but i couldn't find something relevant in the forums.

Share this post


Link to post
Share on other sites
Advertisement
There has been many a post on the forums about how taskman is NOT the best way to judge your application's memory usage.
Try some custom allocator solutions like fluid studio's memory manager.

Share this post


Link to post
Share on other sites
Quote:
Original post by ToohrVyk
How do you fill the vertex and index buffers?

What do you mean? I'm using VBOs and i have tried both Map/UnMap and BufferSubData. There is no difference in the final memory footprint.
One little weird thing is that when mapping the buffer for writing, there are some allocations (the memory increases) which seems logical because the drivers may want to map the buffer in system memory for faster copies (this is what GLExpert says). But when UnMapping the buffer the allocated memory seems to be kept.

For vertex buffers i have one big VBO (4MB) and i segment that into smaller parts. For index buffers i have one VBO for each one of them.

May this be the problem?

EDIT:
Quote:
Original post by KulSeran
There has been many a post on the forums about how taskman is NOT the best way to judge your application's memory usage.
Try some custom allocator solutions like fluid studio's memory manager.

I suspected that, that's why i'm trying to find an other solution to the problem (before writing my own memory allocators). One question though. Isn't this (what the task manager reports) an estimate on what's going on? What about the page file? Isn't, what's reported, correct?

HellRaiZer

Share this post


Link to post
Share on other sites
I just compiled the engine with Fluid Studios memory manager.
I did a memory dump just after engine initialization (no scene loaded) and one just before shuting down the engine (where everything used at run-time are still alive).

After initialization it reported :

TOTALS
-------
Allocation unit count: 1,381
Reported to application: 43,735 ( 42.71K)
Actual total memory in use: 87,927 ( 85.87K)
Memory tracking overhead: 44,192 ( 43.16K)

PEAKS
------
Allocation unit count: 1,437
Reported to application: 51,316 ( 50.11K)
Actual: 94,841 ( 92.62K)
Memory tracking overhead: 43,525 ( 42.50K)

ACCUMULATED
------------
Allocation unit count: 2,387 ( 2.33K)
Reported to application: 111,210 ( 108.60K)
Actual: 187,594 ( 183.20K)

UNUSED
-------
Memory allocated but not in use: 9,104 ( 8.89K)



Before shutdown:

TOTALS
-------
Allocation unit count: 21,031
Reported to application: 4,287,021 ( 4.09M)
Actual total memory in use: 4,960,013 ( 4.73M)
Memory tracking overhead: 672,992 ( 657.22K)

PEAKS
------
Allocation unit count: 21,034
Reported to application: 12,574,474 ( 11.99M)
Actual: 13,111,498 ( 12.50M)
Memory tracking overhead: 537,024 ( 524.44K)

ACCUMULATED
------------
Allocation unit count: 106,898 ( 104.39K)
Reported to application: 202,803,335 ( 193.41M)
Actual: 206,224,071 ( 196.67M)

UNUSED
-------
Memory allocated but not in use: 39,008 ( 38.09K)



The reason why i did just those two dumps is that from the run-time phase to the shutdown nothing is being freed. Not something big at least. At the time the engine is about to shutdown, the task manager reports 128MB.

Is my app really using 128MB or 4MB? I'm a little confused with this, so any help is appreciated.
What's the accumulated output? Is this the total memory allocated by my app, if i didn't free any of it?

Thanks again.

HellRaiZer

Share this post


Link to post
Share on other sites
You do realize that every byte you allocate for a VBO is backed by an equivalent amount of system memory, right?

Share this post


Link to post
Share on other sites
I wouldn't worry about system memory too much.

You will also notice, it will use a different amount of memory on each computer. Mostly computers with 2+ gigs it will use 2x more system memory because it can.
(explorer.exe uses 20 megs on my computer, my friends, it uses 40 because he has 2 gigs of ram, and his system cache is around 400 megs, mines about 220)

As long as your not killing the machine so it has to start using the page file.

I pretty much only use system memory to see if I have memory leaks, thats about the only thing its good for. I leave my game on over night stress testing and i come back and compare the fps and memory.

Share this post


Link to post
Share on other sites
I dont have anything to say about your problem exactly but I just want to say a very strange thing that happens here.

When I open a program, for example 3dsmax, the Task Manager shows me it is cosuming approx 160 MB. Then I minimize the 3dsmax window and the Task Manager shows me a bit more than 2 MB. Finally, I maximize the 3dsmax window and it shows me only 25 MB...The Task Manager seems to be too crazy...

Share this post


Link to post
Share on other sites
Another thing to note is that taskman is reporting the memory in pages allocated to your application.
if you allocate 20 pages of 16byte objects, then delete all but 20 of those objects you run the chance
of one object being on each page, so the page doesnt get deallocated.
So memory fragmentation causes taskman to show you that you are using tones of pages of ram, but
at the same time it is very sparsely allocated within your application.

Share this post


Link to post
Share on other sites
Hi again and thanks for the replies.

Quote:
Original post by Promit
You do realize that every byte you allocate for a VBO is backed by an equivalent amount of system memory, right?

The whole time? I would suspect something like this to happen when i'm reading/writing data from/to a VBO. I'll have that in mind. Thanks. But even if this is the case, all VBOs (vertex + index buffers) take up only 8MB (4MB for vb and the rest for ib) of memory. So this must not be what's causing the huge gap between what the taskman says and the memory manager.

Quote:
Original post by ViperG
I wouldn't worry about system memory too much.

You will also notice, it will use a different amount of memory on each computer. Mostly computers with 2+ gigs it will use 2x more system memory because it can.
(explorer.exe uses 20 megs on my computer, my friends, it uses 40 because he has 2 gigs of ram, and his system cache is around 400 megs, mines about 220)

I think this only happens for Windows (explorer.exe and other windows' apps). I don't think this applies to my app, or it does?

Quote:
As long as your not killing the machine so it has to start using the page file.

That's my point. If taskman is reporting too much memory for my app, i would expect to start using the page file after some time.

Quote:
Original post by KulSeran
Another thing to note is that taskman is reporting the memory in pages allocated to your application.
if you allocate 20 pages of 16byte objects, then delete all but 20 of those objects you run the chance
of one object being on each page, so the page doesnt get deallocated.
So memory fragmentation causes taskman to show you that you are using tones of pages of ram, but
at the same time it is very sparsely allocated within your application.

This seems to be the "problem". What can i do in case to prevent such behavior? Should i use a custom memory allocator, which allocates memory once (say 16MB) and then get from that small chunks for the objects? Will this helps somehow?

Thanks for the helpful replies.

HellRaiZer

Share this post


Link to post
Share on other sites
in taskmanager:

view > select columns > (check: memory usage && Virtual Memory Size && peak memory usage)

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement