# [.net] Memory profiling a CPU-intensive app.

This topic is 5195 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

Okay, so I'm writing a .Net application for a research project, and I've run into the problem of memory consumption. The program runs along at a nice steady pace, consuming just over 40 megs of ram, which is about what I expect for the application. However, after a few minutes running, the memory usage suddenly and for no apparent reason starts to climb extremely rapidly and doesn't stop until the program crashes, complaining that it is out of memory. I tried breaking out the good ol' CLR profiler to see what's up, but my program is extremely CPU-intensive, and using the CLR profiler slows it down to a crawl, meaning that it will probably take many hours if not days to run, which makes using the CLR profiler extremely impracticle. So does anyone know of a way to examine what's going on in my app's memory without slowing it down to a crawl? (I tried turning off all the profiling features but to no avail) All I want is a snapshot of what it looks like just before it crashes. Thanks for any tips! - Fuzz

##### Share on other sites
I just tried the .NET 2 beta 2 CLR profiler and it is much better! You can see a timeline of allocations even! That would be useful for you - if you don't mind running your app for a day...

Do you make a lot of allocations in your program? If not, I don't see why only profiling allocations should slow it down considerably.

My program, a 3D game is extremely CPU intensive too but is hardly slowed at all by profiling calls and allocations at the same time.
However, I suppose it might be pixel shader bound rather than limited by the CPU but task manager says 100% so I guess that wouldn't make much difference.

Andrew <-- hoping he's not wrong

##### Share on other sites
Quote:
 I just tried the .NET 2 beta 2 CLR profiler and it is much better! You can see a timeline of allocations even! That would be useful for you - if you don't mind running your app for a day...

Yeah, that's the one I have. It does exactly what I need it to do, only too slow.

Quote:
 Do you make a lot of allocations in your program? If not, I don't see why only profiling allocations should slow it down considerably.

No, I don't really make very many allocations (though clearly I do when the mem usage jumps to 1 gb). I tried turning off both memory profiling and call profiling, but it still ran extremely slowly. I really only need to see the heap dump of the app when the memory has skyrocketed.

Quote:
 My program, a 3D game is extremely CPU intensive too but is hardly slowed at all by profiling calls and allocations at the same time.However, I suppose it might be pixel shader bound rather than limited by the CPU but task manager says 100% so I guess that wouldn't make much difference.

Though my app *IS* an MDX program, but it is very much CPU bound since it performs a lot of physics and AI.

I'll recheck my profiling settings. The fact that it's over 100 times slower with all profiling options disabled sounds strange to me.

##### Share on other sites
Quote:
 Original post by Holy FuzzI'll recheck my profiling settings. The fact that it's over 100 times slower with all profiling options disabled sounds strange to me.

Actually, 100 times is overexagarated, but it's very possible. I profiled one or two applications in the past, and during profiling, the application was SLOW. The fact that is it slower, is that every function call is timed and measured. This adds in alot of additional code. This code makes the application pretty slow.

Can't you run the application run for a while and see what happens? Also, why not let it crash? You can catch the exception, and print out a stacktrace. You can find where the allocation went wrong.

I had an InvalidCallException last night. Using StackTrace I figured out the exception was thrown from DrawPrimitives().

Toolmaker

##### Share on other sites
Hi mate, Im working in research too, using Native C++, Managed C#, GDI+ and DirectX to do a bit of calculations, display some data, blah blah..

Anyway, I had exactly the same problem as you - a piece of software with some pretty tight loops and not excessive memory allocation did its job, then halfway through, memory usage jumped to max (1G + 1G virtual), and the computer crashed. Anyway, I found the problem and sorted it now - its .NET's rubbish garbage collector.

Basically, in 'office' style apps, the pace of CPU usage is too slow to cause the garbage collector to lock up. In a CPU intensive app, the GC cant do its job, as it relies on windows messaging to clean up in idle time.

The solution? DoEvents. Yes, a well placed Application.DoEvents() inside a tight loop allows the GC to do its dirty work.

To get more control over your deallocation, apply the following principles to your code wherever you have problems:

eg:

void myMegaCalculation()
{
// Start horrendous calculation
double [] megaBuffer = new double[hugeNumber];

for (i = 0, j = 0; i < largenumber; i++, j++))
{
megaCalculation1();
megaCalculation2();
displayRoutine();

// Might be necessary, might not. try it and see
if (j == 100)
{
Application.DoEvents();
j = 0;
}

}

// End calculation and clean up
GC.Collect(); // Force the Garbage Collector to get out of bed and collect
Application.DoEvents(); // Give GC some windows down-time to do its job

}

...

Please note, too much GC.Collect() and DoEvents will make your program mega-slow. Like I said. A couple of well placed collections & doevents will do fine. Too many, and your program crawls.

• 11
• 23
• 36
• 16
• 75