Jump to content
  • Advertisement
Sign in to follow this  
Holy Fuzz

[.net] Memory profiling a CPU-intensive app.

This topic is 4830 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Okay, so I'm writing a .Net application for a research project, and I've run into the problem of memory consumption. The program runs along at a nice steady pace, consuming just over 40 megs of ram, which is about what I expect for the application. However, after a few minutes running, the memory usage suddenly and for no apparent reason starts to climb extremely rapidly and doesn't stop until the program crashes, complaining that it is out of memory. I tried breaking out the good ol' CLR profiler to see what's up, but my program is extremely CPU-intensive, and using the CLR profiler slows it down to a crawl, meaning that it will probably take many hours if not days to run, which makes using the CLR profiler extremely impracticle. So does anyone know of a way to examine what's going on in my app's memory without slowing it down to a crawl? (I tried turning off all the profiling features but to no avail) All I want is a snapshot of what it looks like just before it crashes. Thanks for any tips! - Fuzz

Share this post


Link to post
Share on other sites
Advertisement
I just tried the .NET 2 beta 2 CLR profiler and it is much better! You can see a timeline of allocations even! That would be useful for you - if you don't mind running your app for a day...

Do you make a lot of allocations in your program? If not, I don't see why only profiling allocations should slow it down considerably.

My program, a 3D game is extremely CPU intensive too but is hardly slowed at all by profiling calls and allocations at the same time.
However, I suppose it might be pixel shader bound rather than limited by the CPU but task manager says 100% so I guess that wouldn't make much difference.

Andrew <-- hoping he's not wrong

Share this post


Link to post
Share on other sites
Quote:
I just tried the .NET 2 beta 2 CLR profiler and it is much better! You can see a timeline of allocations even! That would be useful for you - if you don't mind running your app for a day...


Yeah, that's the one I have. It does exactly what I need it to do, only too slow.

Quote:
Do you make a lot of allocations in your program? If not, I don't see why only profiling allocations should slow it down considerably.


No, I don't really make very many allocations (though clearly I do when the mem usage jumps to 1 gb). I tried turning off both memory profiling and call profiling, but it still ran extremely slowly. I really only need to see the heap dump of the app when the memory has skyrocketed.

Quote:
My program, a 3D game is extremely CPU intensive too but is hardly slowed at all by profiling calls and allocations at the same time.
However, I suppose it might be pixel shader bound rather than limited by the CPU but task manager says 100% so I guess that wouldn't make much difference.


Though my app *IS* an MDX program, but it is very much CPU bound since it performs a lot of physics and AI.

I'll recheck my profiling settings. The fact that it's over 100 times slower with all profiling options disabled sounds strange to me.

Share this post


Link to post
Share on other sites
Quote:
Original post by Holy Fuzz
I'll recheck my profiling settings. The fact that it's over 100 times slower with all profiling options disabled sounds strange to me.


Actually, 100 times is overexagarated, but it's very possible. I profiled one or two applications in the past, and during profiling, the application was SLOW. The fact that is it slower, is that every function call is timed and measured. This adds in alot of additional code. This code makes the application pretty slow.

Can't you run the application run for a while and see what happens? Also, why not let it crash? You can catch the exception, and print out a stacktrace. You can find where the allocation went wrong.

I had an InvalidCallException last night. Using StackTrace I figured out the exception was thrown from DrawPrimitives().

Toolmaker

Share this post


Link to post
Share on other sites
Hi mate, Im working in research too, using Native C++, Managed C#, GDI+ and DirectX to do a bit of calculations, display some data, blah blah..

Anyway, I had exactly the same problem as you - a piece of software with some pretty tight loops and not excessive memory allocation did its job, then halfway through, memory usage jumped to max (1G + 1G virtual), and the computer crashed. Anyway, I found the problem and sorted it now - its .NET's rubbish garbage collector.

Basically, in 'office' style apps, the pace of CPU usage is too slow to cause the garbage collector to lock up. In a CPU intensive app, the GC cant do its job, as it relies on windows messaging to clean up in idle time.

The solution? DoEvents. Yes, a well placed Application.DoEvents() inside a tight loop allows the GC to do its dirty work.

To get more control over your deallocation, apply the following principles to your code wherever you have problems:

eg:

void myMegaCalculation()
{
// Start horrendous calculation
double [] megaBuffer = new double[hugeNumber];

for (i = 0, j = 0; i < largenumber; i++, j++))
{
megaCalculation1();
megaCalculation2();
displayRoutine();

// Might be necessary, might not. try it and see
if (j == 100)
{
Application.DoEvents();
j = 0;
}

}

// End calculation and clean up
GC.Collect(); // Force the Garbage Collector to get out of bed and collect
Application.DoEvents(); // Give GC some windows down-time to do its job

}


...

Please note, too much GC.Collect() and DoEvents will make your program mega-slow. Like I said. A couple of well placed collections & doevents will do fine. Too many, and your program crawls.

Also check out a program called ANTS Profiler (google it). It profiles memory and execution time of all .NET code. Awesome product, 14 day trial, or $300 for full version I think. Using this helped me to find out it wasn't my program that wasn't releasing memory, but some internal Windows routines that were getting clogged up by so many calcs.

Good luck!!

Andy B

-------------------------------------
Edit, with regards to 3D game programming, in .NET you are forced to use the old Render(); DoEvents loop to allow Garbage Collecting. ie:

void Main (void)
{
// Create a new app
using (myApp = new myCustomApplication())
{
myApp.Show();

// Super tight loop
while(myApp.Created)
{
// all the business happens here
myApp.Render();
// allow some time for Windows to keep up
Application.DoEvents();
}
}
}

Also try threading to split components up into various sections. ie: rendering thread, physics thread, main thread etc... The problem most likely lies with the high CPU stalling the GC. Try ways to reduce or share the CPU loading through some heavy optimisation and well-placed message handling.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
You could try Memprofiler (http://memprofiler.com). I've used it in my work in the past and it has worked quite well.

Share this post


Link to post
Share on other sites
@Toolmaker: I'll try that out... Hmm, it doesn't even crash on my slow laptop here at work. It just sits thrashing the harddrive constantly. The mem usage does jump to 150 mb or so, but then settles back down to around 40. I'll try it out on my fast desktop when I get home.

@Andy: What you say sounds strange to me, from my knowledge of the CLR. It's my understanding that the GC will be invoked when an app runs out of memory given to it by the OS, meaning that allocations of new objects can trigger the GC, in which case you shouldn't have to call DoEvents to make the GC do its job. Besides, then how does the GC work in non-windowed apps? I do call GC.Collect() at reasonably regular intervals (my app runs a lot of independant simulations, and I call Collect between each one since, in this case, I know better than the GC when it is a good time to do a large collection).

However, the evidence of my problem certainly fits what you are describing. My app is designed to perform logic updates at a certain rate and to render/DoEvents as fast as it can when not performing updates. Sometimes it takes too long to perform updates, and the application gets behind where it thinks its supposed to be and stops rendering or calling DoEvents while it trys to catch up. During this period is when the mem usage appears to go crazy.

@AP: thanks, i'll check that out.

Share this post


Link to post
Share on other sites
You can use the performance counters that the Microsoft .NET Runtime team ship with .NET to view many different aspects about the garbage collector.

Type perfmon.exe at the command prompt, press the + sign button on the toolbar, find the .NET CLR Memory performance object, and add the counters you want. Voila, you can see a visual graph that depicts the allocation profile of your application.

Hope that helps!

Graham

Share this post


Link to post
Share on other sites
Whoops - I thought I'd already posted this but apparantly not -

@Holy Fuzz @Andy - yes, the GC runs on a separate thread so using lots of CPU shouldn't prevent collections.

Sad to see your problem hasn't been solved yet :(

Does the crash happen at the same time each time you run your app? (That might not count if you use the system time or random numbers) Have you said what your app does?

Share this post


Link to post
Share on other sites
Alright, problem solved guys! It turns out that it has nothing to do with DoEvents or the GC not running properly. It was simply a bug in my code that only showed up when the CPU becomes overloaded that caused resource-allocation heavy code to be called in effectively in infinite loop.

So case closed! Thanks all.

- Fuzz

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!