• Announcements

    • khawk

      Download the Game Design and Indie Game Marketing Freebook   07/19/17

      GameDev.net and CRC Press have teamed up to bring a free ebook of content curated from top titles published by CRC Press. The freebook, Practices of Game Design & Indie Game Marketing, includes chapters from The Art of Game Design: A Book of Lenses, A Practical Guide to Indie Game Marketing, and An Architectural Approach to Level Design. The GameDev.net FreeBook is relevant to game designers, developers, and those interested in learning more about the challenges in game development. We know game development can be a tough discipline and business, so we picked several chapters from CRC Press titles that we thought would be of interest to you, the GameDev.net audience, in your journey to design, develop, and market your next game. The free ebook is available through CRC Press by clicking here. The Curated Books The Art of Game Design: A Book of Lenses, Second Edition, by Jesse Schell Presents 100+ sets of questions, or different lenses, for viewing a game’s design, encompassing diverse fields such as psychology, architecture, music, film, software engineering, theme park design, mathematics, anthropology, and more. Written by one of the world's top game designers, this book describes the deepest and most fundamental principles of game design, demonstrating how tactics used in board, card, and athletic games also work in video games. It provides practical instruction on creating world-class games that will be played again and again. View it here. A Practical Guide to Indie Game Marketing, by Joel Dreskin Marketing is an essential but too frequently overlooked or minimized component of the release plan for indie games. A Practical Guide to Indie Game Marketing provides you with the tools needed to build visibility and sell your indie games. With special focus on those developers with small budgets and limited staff and resources, this book is packed with tangible recommendations and techniques that you can put to use immediately. As a seasoned professional of the indie game arena, author Joel Dreskin gives you insight into practical, real-world experiences of marketing numerous successful games and also provides stories of the failures. View it here. An Architectural Approach to Level Design This is one of the first books to integrate architectural and spatial design theory with the field of level design. The book presents architectural techniques and theories for level designers to use in their own work. It connects architecture and level design in different ways that address the practical elements of how designers construct space and the experiential elements of how and why humans interact with this space. Throughout the text, readers learn skills for spatial layout, evoking emotion through gamespaces, and creating better levels through architectural theory. View it here. Learn more and download the ebook by clicking here. Did you know? GameDev.net and CRC Press also recently teamed up to bring GDNet+ Members up to a 20% discount on all CRC Press books. Learn more about this and other benefits here.
Sign in to follow this  
Followers 0
fir

memory map questions

7 posts in this topic

when watching memory map page in olly debug (very fine thing)

 

[attachment=22617:mm.jpg]

 

some things are clear for example loaded dlls and its sections (those blocks that are described as Image),  (one thing unclear why here are loaded .realoc data as it seem not needed in memory, i dont know) - but the other ones are unclear , i mean those described as Map and Priv - does maybe someone know what it is, what it could be, why it is needed - i would wery like to get know whats that

 

also yet one intriguing question - i see there .data sections for each modules but data is only static and non zero initialized data ,I wonder where in such blocks lays mallocked data or zero initialized static data.. Is there some way to get know where each module mallocked (or static zero initialized data) lays here?

Edited by fir
0

Share this post


Link to post
Share on other sites

Uninitialized data has been put in BSS (Base Storage Segment) for some hundred years of computer technology.

malloced data is taken from a block that gets requested from the OS. sbrk() and brk() are the ones.

1

Share this post


Link to post
Share on other sites

I guess .reloc could probably be reclaimed - I don't think an image (exe/dll) will be moved again after it's loaded.  Though I've never seen a process reclaim .reloc pages for other purposes yet.

 

As far as I can tell from the picture, "Map" stands for memory-mapped-file.  Memory mapped files are an operating system feature that let you map a file or a portion of a file directly to memory - reads and writes to the memory will impact the file without having to fread/fwrite/etc.

 

All of the other stuff (marked as "Priv") are anything the process has allocated dynamically.  These could be heaps, stacks, JITted code (if you see any pages outside of the image regions that are marked as execute, it's JITted or injected code), or whatever else the program was written to allocate.  It depends on the program (which language, runtime library, manual memory management, etc it uses).

 

The BSS pages in your picture are expanded in the middle of the image pages by the loader.  They are fixed in that position because the executable was built to expect them at that location.

 

The pages marked as "RW Guard" are guard pages used to detect stack overflows by triggering a special exception if they are accessed.

alright, tnx

 

i overlooked this 70 MB .bss sections (it seemed that it not showed previously but probably i was blind) - seem to be a bit large, it seemed to me that i got more like 20 MB static arrays here but again probably im blind in counting (this app is my pure c + winapi rasterizer small 300 kb exe with no redistribuables)

 

system dlls seem to not have .bss sections (it seemed strange for me but maybe they are written such way - mallocs instead bss)

 

stack seem to have 16 kb only but there is adress space for potential 2 MB 

 

1. those "Map" blocks still confuse me, there ara about 20 of them mostly small, most seem to be data (some read some radwrite) but some are executable (and mysterious - not described)

 

do  such memory mapped meand more like real disk file that is mapped or this is more like shared physical ram blocks ? are those really disk files? this is confucing - maybe someone is able to clear it a bit

 

2. thise privs also confuse me yet - i wonder if those are just mallocked buffers - sad there is no description (if so) which module mallocked them 

(i mean if loaded dll's if some loader external to those dlls (like stack was probably allocked by some loader or what) or my prog

 

I could probably make same experiment by logging my mallocks adresses and comparing them to such blocks.

 

maybe those privs together make juz a mallocked heap? if someone could add something to this picture it would be fine 

 

ps. some privs here are marked executable but as this is moderately simple winapi app there should be no jitted or injected code.. so i dont know how to explain this

Edited by fir
0

Share this post


Link to post
Share on other sites

system dlls seem to not have .bss sections (it seemed strange for me but maybe they are written such way - mallocs instead bss)


If the uninitialized data is small, the compiler might decide to include it in the initialized data section instead.
 

stack seem to have 16 kb only but there is adress space for potential 2 MB


The stack overflow handling behavior can expand the stack on the fly (if it's set up to do so) and resume execution gracefully if the stack was expanded safely.
 

1. those "Map" blocks still confuse me, there ara about 20 of them mostly small, most seem to be data (some read some radwrite) but some are executable (and mysterious - not described)
 
do  such memory mapped meand more like real disk file that is mapped or this is more like shared physical ram blocks ? are those really disk files? this is confucing - maybe someone is able to clear it a bit


I haven't personally used memory mapped files more than once. I believe you can use them to share data between two processes with or without "real" disk files, but I've never actually tried that.
 

2. thise privs also confuse me yet - i wonder if those are just mallocked buffers - sad there is no description (if so) which module mallocked them 
(i mean if loaded dll's if some loader external to those dlls (like stack was probably allocked by some loader or what) or my prog

I could probably make same experiment by logging my mallocks adresses and comparing them to such blocks.
 
maybe those privs together make juz a mallocked heap? if someone could add something to this picture it would be fine 
 
ps. some privs here are marked executable but as this is moderately simple winapi app there should be no jitted or injected code.. so i dont know how to explain this


"Priv" probably stands for "Private bytes" (which is what a few process memory reporting tools on Windows call a process' memory usage).

As far as I know, here's how it works:

- VirtualAlloc is what allocates the pages you see in your screenshot, and VirtualProtect sets their flags. However, a heap implementation is free to make as many different page allocations as it wants to, even if they are not all in one block of contiguous address space. Since page sizes are a fixed size (usually 4096 bytes for a typical 32-bit Windows process), there may be some unused space in them that the runtime library's memory manager hasn't specifically made part of its heap yet.

The different DLLs in your process are free to share a heap, or use multiple separate heaps as well. They avoid overlapping with each other because VirtualAlloc won't let you allocate pages which are already allocated.

As far as your unknown Priv+execute pages go, any third-party applications on your computer may install system hooks which cause DLLs to be loaded into your processes even if they aren't normally part of your program. Mice and keyboard vendors, for example, may with software that will load various DLLs into (nearly) every process in order to handle special shortcut keys or gestures. It might also be something Olly is doing (I haven't used Olly - I have my own tool to show the same kinds of information, except I know for certain that it does not inject anything). Edited by Nypyren
2

Share this post


Link to post
Share on other sites

 

system dlls seem to not have .bss sections (it seemed strange for me but maybe they are written such way - mallocs instead bss)


If the uninitialized data is small, the compiler might decide to include it in the initialized data section instead.
 

stack seem to have 16 kb only but there is adress space for potential 2 MB


The stack overflow handling behavior can expand the stack on the fly (if it's set up to do so) and resume execution gracefully if the stack was expanded safely.
 

1. those "Map" blocks still confuse me, there ara about 20 of them mostly small, most seem to be data (some read some radwrite) but some are executable (and mysterious - not described)
 
do  such memory mapped meand more like real disk file that is mapped or this is more like shared physical ram blocks ? are those really disk files? this is confucing - maybe someone is able to clear it a bit


I haven't personally used memory mapped files more than once. I believe you can use them to share data between two processes with or without "real" disk files, but I've never actually tried that.
 

2. thise privs also confuse me yet - i wonder if those are just mallocked buffers - sad there is no description (if so) which module mallocked them 
(i mean if loaded dll's if some loader external to those dlls (like stack was probably allocked by some loader or what) or my prog

I could probably make same experiment by logging my mallocks adresses and comparing them to such blocks.
 
maybe those privs together make juz a mallocked heap? if someone could add something to this picture it would be fine 
 
ps. some privs here are marked executable but as this is moderately simple winapi app there should be no jitted or injected code.. so i dont know how to explain this


"Priv" probably stands for "Private bytes" (which is what a few process memory reporting tools on Windows call a process' memory usage).

As far as I know, here's how it works:

- VirtualAlloc is what allocates the pages you see in your screenshot, and VirtualProtect sets their flags. However, a heap implementation is free to make as many different page allocations as it wants to, even if they are not all in one block of contiguous address space. Since page sizes are a fixed size (usually 4096 bytes for a typical 32-bit Windows process), there may be some unused space in them that the runtime library's memory manager hasn't specifically made part of its heap yet.

The different DLLs in your process are free to share a heap, or use multiple separate heaps as well. They avoid overlapping with each other because VirtualAlloc won't let you allocate pages which are already allocated.

As far as your unknown Priv+execute pages go, any third-party applications on your computer may install system hooks which cause DLLs to be loaded into your processes even if they aren't normally part of your program. Mice and keyboard vendors, for example, may with software that will load various DLLs into (nearly) every process in order to handle special shortcut keys or gestures. It might also be something Olly is doing (I haven't used Olly - I have my own tool to show the same kinds of information, except I know for certain that it does not inject anything).

 

Here, this is not me that uses this memory map, but the system - if priv are private for  this wirtual space maybe map means blocks shared for all or many wirtual spaces (?)

 

As to private, i made some experiment I was writing before and printed the  adresses of two mallocked buffers I was using in my map and those appeared as a Priv RW here; Wonder if all privs here are just some mallocks 1-1

Yet one thing im not using many mallocks in my program (im using almost exclusively only ststic arrays) Im quite sure that in this program Im using only two reallocks yet thios screenshot is taken at program startup so they was not yet fires so all those privs are probably called by

1) some loader 2) some loaded dll's startups (if those are able to fire such startup thing )

- wonder if there should be an option to do not those static like but manual and deferred loading of those dll-s so i could wath it at runtime and see as more chunks appear... (Im interested in such kind of tings but msdn is usually to much confusing yet for me)

(- as to injections i got avast antivir installed he really injects something even if disabled, once he was killing my app that was hackishly stripped for size as he found it as malware)

Yet i wonder if is dll is mallocking data does such dll hold it out of my wirtual process space or it is in it? such dll data is probably not needed to be seen by my process (at least in 99% of cases) but those picture seem to suggest that it is loaded in the same wirtual space - not sure

 

Can you say yet what you mean by many heaps? each such heap is in different wirtual space or those heaps lays in one wirtual space but are defined over some distinct blocks, or what? tnx for info, if i find some more info i cand extend this picture yet in some details, very interesting things, much usefull for some kind of things (like taking care whats going on)

0

Share this post


Link to post
Share on other sites

Wonder if all privs here are just some mallocks 1-1


They're not 1-to-1, otherwise you'd have very inefficient use of RAM. For example if you allocated two blocks of 16 bytes apiece, they will probably go in the same Priv page. You wouldn't want to have a separate 4K priv page for each one.

Can you say yet what you mean by many heaps? each such heap is in different wirtual space or those heaps lays in one wirtual space but are defined over some distinct blocks, or what? tnx for info, if i find some more info i cand extend this picture yet in some details, very interesting things, much usefull for some kind of things (like taking care whats going on)


It's implementation dependent (it's up to the memory manager). For example, the .Net Large Object Heap probably puts its pages in a different area than the generational heap(s) to avoid fragmentation on the LOH from impacting the generational heap. The MSVCRT heap can also do the same kind of thing. You'll likely see completely different memory management schemes between different vendors.
1

Share this post


Link to post
Share on other sites

Olly debug builds a virtual memory space so the program being debugged can be viewed.  Every program whether normal,dll, or text will be broken down into a perceived memory space in Olly.  As far as Private goes... It's privileaged memory space and Windows defines it based on it's system memory map.  With this space you must be careful!  This changes on everyone's elses system based on their overall system memory(physically-installed into their system).  Olly is just displaying the code for you regardless. (Nypyren and other's are rite I'm just padding here to help with your understanding of what's going on).  Your memory can also be displayed into 1-byte, 2-byte(word), 4-byte(longword-old motorola 68k wording), etc,etc.  Again, just for reference purposes.  For example, load a picture file into olly and watch what happens. Or, a sound file.  That's the great thing about olly debug; It'll let you debug anything!

 

Essentially, It's all for reference for debugging code.  There is no physical memory addressing going on with olly. In fact, only windows assigns(and reallocates) and it IS different each time because of competing memory space. 

 

And in closing, the 'heap' is sometimes called STACK memory.  And olly does monitor stack calls very good. Be careful not to confuse your program's stack calls with Win32(System) calls.  Especially, late night debugging sessionswacko.png .  You might go into a rabbit hole and not get back out!unsure.png

Edited by Counteractman
1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0