Archived

This topic is now archived and is closed to further replies.

cgoat

Linux memory usage with multithreaded apps

Recommended Posts

cgoat    132
I was wondering if anyone could fill me in on the details of how linux multithreaded applications use up memory. I have a program that has 50 threads. Using "gtop" and looking at the "Memory Usage (resident)" tab shows that my program is using 1,249,456k. That''s over 1 gig. It''s not possible, and I''m wondering how gtop arrives at that number. I''m running on a RedHat 6.2 linux machine, basically out of the box. The program has very large global data, probably on the order of 20-30mb. (It''s declared extern in a globals.h file). My guess was that since threads on linux are treated as separate processes, but those threads share global memory, that the usage report is skewed counting the shared global memory each time for each thread. As a test, I wrote a program that simply starts 50 threads, no global data. gtop reports usage of 100048k for that program. If I allocate 1mb of global data (actually 256000 longs), then start the 50 threads, gtop reports usage of 149136k. It seems to prove my guess, except that if I use up more memory, it doesn''t increase. I tried it with 512000 longs and 1024000 longs and get the same usage (149136k). It doesn''t make sense to me. I just don''t understand how to tell how much memory my program uses. Any help would be appreciated.

Share this post


Link to post
Share on other sites
Null and Void    1088
Per-process memory usage measuring factors all kinds of strange things (video memory, shared memory, memory from libraries, et cetera).

The best way I can think of to measure the memory you''re directly responsible for using is to run ''free'' and look at the system memory in use (make sure to use a value that does not include memory buffers), run your program (and get it into whatever state it should be in), and then run free again (and look at the same value).

It''ll be a little off, so running it two or three times will probably give you the best indication of your program''s effect on memory usage.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster   
Guest Anonymous Poster
If you look at the "Shared Memory:" thing in plain old ''top'' you''ll see a nice big zero.

Shared memory is too difficult to calculate. (difficult in CS terms, i.e. easy to code, takes forever to run.)

the free method is probably the easiest to do.

Share this post


Link to post
Share on other sites