#### Archived

This topic is now archived and is closed to further replies.

This topic is 5501 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

this is sort of an advanced topic, so if you don’t know what a heap is or what cross-boundary memory sharing is, you probably won’t follow me on my dumb explanation. O.k. Here’s the beef. At the moment I am developing a 2D shoot'em'up for about 5 month now. I am also using my preparatory game engine 'nan2D', witch is in a DLL. So the two main modules are:
• the engine (nan2d.dll)
• The game executable (xyzzy.exe) all the rest of junk are game assets (art files, sound files, ...) that have no relationship with my problem. Now before I tell you about my problem, I want you to know that I have been struggling with this particular problem for exactly 8 days now. So before I say any more, I want to tell you that I might ‘pop’ at any minute and start talking out of my ass. Also, I really do hope that ‘we’ (as in all participants I this thread) can get rid of my problem… yes that would be very nice of you if you did have a hand in this. I cant stress enough how grateful I will be if someone helps me! So please bare with me and read about my problem. I am assuming that you know how a DLL works and what it can and can not do, here are some things about it:
• A DLL is always loaded in memory, no matter how many applications use it… there is always one instance of it. Yeh that was only one… and I don’t see how that has to do anything with my problem. I am starting no type nonsense, lets get down to the problem shall we. In my engine, I have functions that take arguments (well duhh), and the engine function can read/write the contents of the variables (in the right scenario). In other word, my engine DLL works just the same as it would as it was statically linked as add-on code modules in my game project. So say that I have a function

void foo(bar& changeMe)
{
/* do some stuff */
changeMe.baz = qwert(); // whatever

}  
So that would be in my engine DLL. And of-cores I would have the correct import/export (ing) declarations before any functions)… I also use the __stdcall calling conversion. Okay, all that will be in my engine. Now in my game code:

bar Somshit;
foo(Someshit); /* whatever it doesn’t have to make any sense */  

LPTSTR strBase = pCfg->GetString( "Explosions","Explosion1SpriteBase" );
LPTSTR strExt = pCfg->GetString( "Explosions","Explosion1SpriteExt" );
N_NEWOBJ( pExplosion1,NAN2DAnimSprite() );
for ( UINT i = 0; i < 30; i++ ) {
TCHAR buf[MAX_PATH];
}
//_asm INT 3

pExplosion1->BuildDefaultAnim();
NAN2DSPRITE_ANIMATION Anim1 = pExplosion1->GetCurrentAnim();
NAN2DSPRITE_ANIMATION Anim2 = pExplosion1->GetCurrentAnim();
Anim2.Reverse();
Anim1.AppendBack( Anim2.GetArray() );
>>> ERROR <<< pExplosion1->SetAnim( Anim1 );
pExplosion1->SetFPS( pCfg->GetFloat( "Explosions","Explosion1SpriteFPS" ) );
pExplosion1->SetRotationCenterNonliner( 0.5f*fGRW,0.5f*fGRH );  
What it does is just load some sprites from the information in an INI file… created a default animation (means it just sequences all the sprites in chronological order and makes an animation out of them)… get the animation that it just made 2 times. One of the animations it got is reversed, or flipped, so that it can be appended to the other clone to make a “bounce” animation (just Ping-Pong’s the animations). Then it sets the ready animation as the current animation in the pExplosion1 (type NAN2DAnimSprite) , witch goes to this code:

/* Created: 2002/05/04 (Sat, May 04) 22:22 */
INT NANCC NAN2DAnimSprite::SetAnim( NAN2DSPRITE_ANIMATION &TheAnim )
{
/* [5/4/2002] See if its already in the list */
>>> ERROR <<< t_2DSpriteAnimArray::iterator result = find(Anim.begin(), Anim.end(), TheAnim);

if (!(result == Anim.end())) {/* [5/4/2002] its in there! */
SetAnimByID((*result).dwID);
return ((*result).dwID);
}else {
}
return -1;
}  
witch goes to this code:

// TEMPLATE FUNCTION find

template<class _InIt,
class _Ty> inline
_InIt find(_InIt _First, _InIt _Last, const _Ty& _Val)
{	// find first matching _Val

for (; _First != _Last; ++_First)
if (*_First == _Val)
break;
return (_First);
}  
Then it just goes down the cryptic stack of STL function calls… Scary shit, I know. “So now you know” what I had to go through in the past 8 days . I joke you not, this is making me slowly go insane! In conclusion: I am not asking for you to tell me how to fix my code and just, I rather would like you to point me to a good resource on DLL programming and some place where I can find out EVERYTHING I need to know about making a DLL, the things that I can and can not do. I also really need to know what is the heap and how to use it wisely, please someone point me in the right direction. One more thing, I like to fix my own problems and dig myself out of my own grave so that is why I am asking you not to fix my code, but to point me in the right direction… but if you know your stuff about all this (memory, heap allocation, DLL) and consider yourself worthy of helping me fix this problem, then please tell me and I will strip the data files (required to get to the point in execution where the error occurs) to almost nothing (like 2x2x8 sprites), and send you the files required to compile everything in MSVS .NET (7.zero) so you can go over use your knowledge in all this to fix or help me fix the problem. I AM using Microsoft Visual Studio .NET! o_0 And most importantly, love yourself and you family… enjoy black history month [edited by - aftermath on January 24, 2003 7:19:33 PM]

##### Share on other sites
quote:
Original post by AfTeRmAtH
this is sort of an advanced topic, so if you don’t know what a heap is

heap is covered in compsci 2.
quote:

or what cross-boundary memory sharing is, you probably won’t follow me on my dumb explanation.

i must've missed "cross-boundary" part of this post. anyway...
quote:

A DLL is always loaded in memory, no matter how many applications use it… there is always one instance of it.

that was the case in win16. in win32, all dlls share their code segments, but unless you make the data segments shareable, they will be private to processes that uses the dll. under win9x, this would be accomplished by duplicating data segments with readwrite protection for every process, and under nt, data segments will have copy-on-write protection.
quote:

The memory for “Somshit” is allocated in the application heap (if I am pondering right), and an address is sent to the function when I call it (if I did not send the address way, with the &, then the “Somshit” variable from the application space would be copied to the DLL application space just for the function context, then destroyed when the context ends (begin={, end=})).

there is no such thing as "dll address space". dll is using the address space of whatever process it is loaded in. in general, process objects (not threads, not modules, etc.) own most resources.
quote:

which requires that the DLL send data to the calling application heap.

if you are allocating memory is dll with new/malloc and freeing it in exe with delete/free, you need to watch out for your c runtime settings. by default, exe and dll each get their own private little singlethreaded statically linked c runtime. this means that pointer allocated with new or malloc1 in one of them doesn't make any sense to the other2, as far as crt is concerned. it's a valid memory block in that you can read and write to it, but the other c runtime didn't allocate it, and thus doesn't know how to free it. the possible solutions are:
- use multithreaded dll runtime for all modules in your project. this way there is only one copy of the c runtime for the entire process, and it will know how to handle everything from all modules.
- provide resource cleanup functions in the dll that free memory that's allocated by the dll. in other words, if dll allocated a memory block, pass it to the dll to free it.

1 this also holds true for other crt objects, such as file handles, FILE * pointers, and so on.
2 if new/malloc are thin wrappers around HeapAlloc and don't add any of their data to memory allocated by heapalloc, as seems to be the case for retail c runtime, then theoretically pointer allocated by one c runtime should be freeable by any other c runtime, as there is only one win32 heap for your process (in contrast to potentially many crt "heaps").
quote:

The problem that I am having is that the application is doing some vary strange shit at runtime.
An example of it would be:

use watch/variables window to snoop around your data, and check if anything is corrupt. there isn't much to go on from the information you've provided. if your program is crashing while attempting to access supposedly perfectly valid data, you may have trashes your stack and/or memory earlier. global data breakpoints are helpful, but these are some of the nastier crashes you can get.

you can also have debug information in release builds; just enable the corresponding options in c++ and linker settings. this will not make your program any slower or bigger, as debug info is stored separately in pdb files, although debugging optimized code can be fun.
quote:

I joke you not, this is making me slowly go insane!

i believe you.
quote:

I rather would like you to point me to a good resource on DLL programming and some place where I can find out EVERYTHING I need to know about making a DLL, the things that I can and can not do.

msdn and google have always worked for me, along with Jeffrey Richter's "advanced windows", which is now called "programming applications." excellent book if you want to know win32 up close and personal.
quote:

I also really need to know what is the heap and how to use it wisely, please someone point me in the right direction.

heap is merely a piece of memory from which you can grab small chunks with functions like malloc. a big issue with heaps is heap fragmentation, which is when you get many small free blocks throughout the heap, but few or none large ones. in this scenario, successively allocating memory does not give you memory blocks that are close together physically, which kills the cache(s) on your processor. if your heap gets fragmented enough, you won't be able to allocate big objects even though there is enough combined free space in it for such allocations.

one way of dealing with fragmentation problems is to have several heaps, each for a certain type(s) of objects. for instance, you can do all small allocations using new and delete, but if you need large memory blocks, let's say several kb in size, you allocate them from another heap (created with HeapCreate, allocate with HeapAlloc). if you have even bigger requests, ranging in hundreds of kilobytes or in megabytes, it might be better to use virtual memory instead (VirtualAlloc). virtual memory allocation functions don't use a specific block of memory that they have to grow every now and then for the allocations; instead, they give you memory straight from your address space, eliminating overhead involved in maintaining the heap. the downside is that VirtualAlloc rounds requests up to the processor page size (4 kb for x86), so using VirtualAlloc for allocations smaller than dozens of kilobytes is impractical.
quote:

One more thing, I like to fix my own problems and dig myself out of my own grave so that is why I am asking you not to fix my code, but to point me in the right direction…

- watch window
- examine any this pointers you have floating around
- data breakpoints
- debug info in release build
- #define DEBUG even for release build, and see what happens
- does this problem occur in debug builds? these have much more error checking thrown in
quote:

but if you know your stuff about all this ... I will strip the data files ... to almost nothing ... and send you the files required to compile everything in MSVS .NET ... so you can go over use your knowledge in all this to fix or help me fix the problem.

mail the stuff my way, i'll look at it.

[edited by - niyaw on January 21, 2003 6:54:00 AM]

##### Share on other sites
quote:

A DLL is always loaded in memory, no matter how many applications use it… there is always one instance of it.

This is not strictly true. You can compile dll''s to run in (at least) two different flavors. The first flavor recieves it''s own data segement everytime it''s loaded (I believe this is the VC default). It can also be compiled to share the same data segement with each instance.

The few dll''s I use that rely on this behavior are written in Delphi, not VC, so I don''t know off-hand what the switches/settings are that affect this. Ought to be in the project settings, or perhaps in the .def file for the dll.

A dll is essentially an .exe that is loaded into the same memory space as a prior exe. The Dll application space is the same as the application that invokes, as far as the data is concerned.
However, the CRT (C run-time) memory allocators create seperate heaps for each one. You are not suppose to alloc in a dll and free in the app nor vice-versa (same for new & delete).

If your dll creates an object, the dll needs a way to destroy it. This is one reason why COM uses reference-counting, and calling ->Release() on the object invokes delete (when the reference count falls to zero). This way the code that does the delete is compiled into the dll, not into the application.

BTW, this is not really a software engineering question, it''s more about a specific debugging or dll issue. However, the ultimate solution to this problem might be a design issue, so I''ll let it slide this time

##### Share on other sites

Wow, thank you for the replies guys, it really means a lot to me. There is a lot to be gone over, so I am just going to refer to the first 2 posts then

To Niyaw:
quote:
heap is covered in compsci 2.

Lucky for me, I am a freshman high-skool student . I don’t know, you probably meant in collage right? The only SompScienc I have and will be having this year are
• Intro to web design (pleh http://www.nan2D.com )
• Computer graphics ( look above)
• Architecture and 3D modeling.

So nothing that has to do with programming, well maybe the intro to web-design thing because we will use HTML (witch is not a programming language by the way). Now back to topic I go!
quote:

I must've missed "cross-boundary" part of this post. anyway…

No a fence, but I have no idea of what you mean by that, please explain
quote:
that was the case in win16. in win32, all DLLs share their code segments…

So does that mean that say… here is the memory:
.............
DLL Space....
.............
..(pr0n).....
.............
Application space.
.............
.............

Does that mean that every time I invoke a function from “DLL Space”, it is always from the same space in memory, no matter what function calls it (I'm guessing that is wrong because then applications would tangle the DLL and make it do unexpected things because there a re to many asynchronous operations at one time); or does every “Application Space” have its local copy of the “DLL Space” (that seams more correct to me)… witch one is right?

quote:
there is no such thing as "dll address space". dll is using the address space of whatever process it is loaded in. in general, process objects (not threads, not modules, etc.) own most resources.

Let me get this right, so like when I have some init. Code in a DLL, and it allocates some objects into memory, its actually just allocating those objects into the application space (identical as if those objects were being allocated by the code in the executable EXE)… am I understanding this correctly?

Also, a CRT (C run-time) is basically a code module (shitload of functions that almost every application uses) this is present when an application is executed. It can be either a DLL or statically linked (meaning that the code from the .lib (in the “lib” directory of the compiler) will be included in my applications .obj files).
Now there are two types of these modules:
• A single threaded module that uses the parent processes (application, DLL, whatever…) memory, and does not access or share any outside memory. The memory that it allocated and de-allocates is only valid within the CRT’s process.
• A multi-threaded module… this one I don’t get a lot about and how it works. I am guessing that it is a CRT module that it present to the calling process and all of its child processes (like the DLL’s that get loaded at run-time (not the ones loaded through LoadLibrary)). Now these things can happen and will be valid:

o The parent process can create (allocate) memory in its heap via “new” or “malloc()”, or whatever… then those functions are invoked to the CRT witch places the newly allocated memory in the CRT heap (witch is “__crtheap” from looking at the source files), witch was created with, assuming, “CreateHeap()”.

o Any child processes (a DLL for example) can allocate memory and it will be taken care of the same way as described above, placed in the same CRT heap.

It looks like I am giving a lecture on the CRT and its types, I am not… I mealy want you (the guys that know this stuff) to tell me what I am getting wrong (and I think that I am getting a lot wrong). Thanks for reading this far too, it know its quite a bit of text, bare with me though.
quote:
are those the errors when you don’t get a fancy shanty dialog box, instead a general MB_OK box telling you of an access violation? If not, what are you talking about?

quote:
you can also have debug information in release builds; just enable the corresponding options in c++ and linker settings. this will not make your program any slower or bigger, as debug info is stored separately in pdb files, although debugging optimized code can be fun.
I am having these sort of crashes in both debug and retail builds.
quote:
I believe you.
so far so good.

quote:
mail the stuff my way, I’ll look at it.
if I just cant find any solution, I will send it to you (gota have MS .net though and DirectX 8.1)

Now to Moderator Magmai :
quote:
This is not strictly true. You can compile dll's to run in (at least) two different flavors. The first flavor receives it's own data segments every time it's loaded (I believe this is the VC default). It can also be compiled to share the same data segement with each instance.

Yes I know about that. It can be accomplished via pragmas and some directive where there is “.segment” and “.data” and whatever… that is not what I want though.

Thanks for the help guys!

Now for more questions (yes I know this Is a bit lengthy ).
What does it mean when my heap is corrupted, and how may I have cause it.
When I compile with the multithreaded debug versions of the CRT, I get the good’ol ASSERT(_Crt…); memory heap checking bs. What might be the problem… the MSDN docs just say that the function validates the heap and crap… but I don’t see what I might be doing that is corrupting it. With the retail versions of the CRT, everything works (well… prior to this 9 day problem that I am having ).

What is the difference (and does it affect my code execution, like does this cause an error if I have it wrong) between a __cdecl function declaration and a __stdcall declaration. I remember having a __fastcall as my default declaration, and that worked too.

And one more thing

When I trace my code, the deadliest part of the code is when the CRT calls AllocHeap (or something like that), and one of its parameters is “__crtheap” meaning that it is allocating some memory onto the heap. Why would this cause errors if the Application and the engine DLL are both using the same CRT heap?

Once again, thank you all for the help!

[edited by - aftermath on January 21, 2003 11:50:55 PM]

##### Share on other sites
quote:
Original post by AfTeRmAtH
Lucky for me, I am a freshman high-skool student .

in high school, the simple new/delete stuff is commonly found in the second semester of AP computer science AB course (www.collegeboard.org). computer science 2 is the college course where i'd expect to find equivalent issues discussed. now, obviously neither of them deals with dynamic-link libraries and the kind of problems you're experiencing.
quote:

No a fence, but I have no idea of what you mean by that, please explain

well, cross-boundary means there has to be a boundary to cross. usually it's process boundary (exchanging data between different running processes, as in two .exe files) or computer boundary (exchange of data over network), or similar; in your case, all operations proceed in one address space, the address space of the process in question (more on this later), and there are no crossed boundaries.
quote:

So does that mean that say… here is the memory:
.............
DLL Space....
.............
..(pr0n).....
.............
Application space.
.............
.............

there's a distinction between virtual memory and physical memory . virtual memory is the kind of memory you're used to. you get virtual memory when you call malloc or new; when you take address of a variable, you get the address in the virtual address space. every running process has its own, private virtual address space . the size of the address space is 4 GB. when an exe is loaded, it looks like this (dots represent unused regions):

.............
.............
.............
exe code and data
.............
.............

now some dlls are loaded, and the address space looks like this:

.............
.............
kernel32.dll code and data
.............
.............
user32.dll code and data
.............
.............
yourdll.dll code and data
someotherdll.dll code and data
.............
exe code and data
.............

note that everything belonging to each dll is stored together in the virtual address space. now let's say you make some heap allocations. the address space will now look like this:

.............
.............
kernel32.dll
.............
default process heap allocated by kernel
storage for object allocated with 'new int'
free heap block
storage for object allocated with 'new CoolClass'
storage for object allocated with 'new char[10000]'
more free heap blocks
.............
.............
game.exe
.............

the important thing to note is that the process object, which is close, but not quite "the exe file", owns the address space. dlls don't own anything, they are merely pieces of memory within the address space that have a name.

now, the virtual memory is not the memory you can find inside your computer. the stuff that's in ram chips is the physical memory . typically, size of your total physical memory (128, 256, 512, ... mb) is much less than combined size virtual address spaces of all running processes (4 gb * 30, 40, ...). how can this be? first, not all blocks of virtual address space are mapped to physical memory (ie, free blocks aren't; they are just marked free, and the operating system can use them, but they aren't occupying any physical memory). two, multiple pieces of virtual memory can reference the same piece of physical memory.

consider a dll called cool.dll that only has code (no data). thus, there are no writes ever to the piece of memory that this dll occupies. now, suppose it is loaded in process A and process B. the layout of process A might be as follows:

.............
some kernel data
.............
cool.dll code
.............
some other data
.............
more data
.............
processA.exe code and data
.............

and layout of process B might be as follows:

.............
some other kernel data
.............
more kernel data
.............
someotherdll.dll code and data
.............
cool.dll code
.............
processB.exe code and data
.............

now, storing two read-only copies of code of cool.dll in your ram is clearly a waste of memory. thus, both cool.dll blocks in the two processes reference the same block of physical memory, which actually contains bits of code of cool.dll (in reality, contiguous virtual memory blocks don't have to correspond to contiguous physical memory blocks, and the way things work is more complicated). thus each process has its own copy of cool.dll, or at least each thinks that way, possibly loaded at different addresses, but there is only one instance of the bits of dll in your physical memory.

Does that mean that every time I invoke a function from “DLL Space”, it is always from the same space in memory, no matter what function calls it

pretty much, which is why dlls are so nice. there are 30 processes in your system, and each of them is referencing kernel32.dll that's 1 mb in size. 29 mb of waste are eliminated with this model.
quote:

(I'm guessing that is wrong because then applications would tangle the DLL and make it do unexpected things because there a re to many asynchronous operations at one time)

not really, because the code is only readable. you can read the same location as much and as often as you want, no harm in that. things get tricky when you want to read and write to memory. which is where we get to data in dlls.
quote:

or does every “Application Space” have its local copy of the “DLL Space” (that seams more correct to me)…

again, there is no such thing as "dll space". dll simply occupies a block of memory that is specially marked so that you know what it is; this block is part of the application address space.

now, consider a dll called win.dll that has both code and data. the code writes to the data that's stored in a dll. the above pictures for virtual memory layouts for cool.dll are exactly the same as virtual memory layouts for win.dll, but the way the dll is stored physically is different. the code segment is still shared among all instances as described above; however, for the data segment, we have two scenarions.

1: the data segment is explicitly marked as "shared". in this case, it's shared in exactly the same way as the code segment; different applications have the dll loaded at potentially different address, but they all reference the same physical piece of memory with dll data. if one application changes the data, other applications will read the changed data. if two applications try to change the data simultaneously, data will most likely be corrupted. this is where thread synchronization steps in, but i'm not going there now.

2: data segment isn't marked as shared; this is the default. now each process thinks it has its own private copy of dll *data*, not just code. if process A modifies a variable in the dll data segment, process B expects to see the old value of the variable. how do we handle this?

win 9x: for each process, a separate copy of the dll data segment is maintained in the *physical memory*. even if most of it is never changed, the data segment is still duplicated. effectively, each process now references a common physical memory block with dll code and a private physical memory block with dll data. this method is simple to implement, but wasteful.

win nt: initially, the entire dll data is stored in a single physical memory block, like the code is. however, once a process tries to write to this block, a copy of the (small) part of it to which the process writes is made specifically for that process. any changes will therefore only be seen by the process which made them. after the write, the writing process will reference (most) of dll data from the shared physical memory block, but it will reference the modified (small) part of dll data from its copied physical memory block. this approach is more memory-efficient, but requires more clock cycles.

i hope this wasn't too confusing; feel free to ask for clarification.
quote:

Let me get this right, so like when I have some init. Code in a DLL, and it allocates some objects into memory, its actually just allocating those objects into the application space (identical as if those objects were being allocated by the code in the executable EXE)… am I understanding this correctly?

mostly. the objects are indeed allocated into the application space. however, problem with new/malloc is that they can attach their own management data to the memory block before giving them to you. that data may be used, for instance, to track memory leaks, or to measure memory load. it's those management blocks that are different from one c runtime to the other. note that you as a user never deal with them, they are internal to new/malloc implementation. again, feel free to ask for clarification.
quote:

Also, a CRT (C run-time) is basically a code module (shitload of functions that almost every application uses) this is present when an application is executed. It can be either a DLL or statically linked (meaning that the code from the .lib (in the “lib” directory of the compiler) will be included in my applications .obj files).

yes. note that eventually the crt code will be in your application address space, be that through a crt dll or through part of your exe file in case of static linking.
quote:

Now there are two types of these modules:
• A single threaded module that uses the parent processes (application, DLL, whatever…) memory, and does not access or share any outside memory. The memory that it allocated and de-allocates is only valid within the CRT’s process.

• almost. the crt doesn't use its parent process for anything. what happens is that this particular instance of crt may add some internal variables to the allocated memory block, and other crts *in the same process* won't make sense of them. keep in mind that we're talking about things occuring entirely within a single process (application).
quote:

• A multi-threaded module… this one I don’t get a lot about and how it works.

• multithreaded static crt functions just like the singlethreaded, but it provides thread synchronization so that if two threads try to access the same data at the same time (ie, ask for some memory to be allocated at the same time) one of them is paused while the other gets its request satisfied. this avoids the problem of two threads modifying the same data simultaneously or one of them reading data that the other one is modifying that i mentioned earlier.

the really different crt is multithreaded *dll* one. with it (provided you use this setting for all dlls and your exe, not just for some) there effectively is only one runtime library for the entire process, in contrast to many crts you'll get if you have dlls linked against any of the two static libraries. while the crt still uses some internal management variables to keep track of its activities, *all modules* in your application reference the same runtime, and it can make sense of the variables that it itself allocated. notice how this is different from static-linked crt scenario, where one crt is asked to make sense of internal variables initialized by another crt *running in the same process*.
quote:

I am guessing that it is a CRT module that it present to the calling process and all of its child processes (like the DLL’s that get loaded at run-time (not the ones loaded through LoadLibrary)).

you seem to be confusing processes and modules (dll modules in particular). stay within one process. there are no child processes around; a child process means you execute another exe file and thus create a new process with a new address space with a new dll set loaded, etc. it has very little to do with loading another dll via LoadLibrary.
quote:

Now these things can happen and will be valid:

o The parent process can create (allocate) memory in its heap via “new” or “malloc()”, or whatever… then those functions are invoked to the CRT witch places the newly allocated memory in the CRT heap (witch is “__crtheap” from looking at the source files), witch was created with, assuming, “CreateHeap()”.

o Any child processes (a DLL for example) can allocate memory and it will be taken care of the same way as described above, placed in the same CRT heap.

this doesn't make much sense. again, child process and a dll have very little in common. stay within one process.

a word on __crtheap: that is the handle to the *application* win32 heap. there is one default heap for every process, and ultimately all memory allocations made with new/alloc are made from it (simplification; actually, this does not always have to be the case, but it's fine for our discussion). again, note that allocating memory with new/malloc from exe and dll:
- gives you memory from the same default win32 process heap (you may even get the same pointer)
- gives you memory that *anything* in your process has read and write access to. this includes code in your exe, code in your dlls, and code in any other dlls that might be present, such as windows dlls.
- differs only by the internal management data that c runtime inserts into the allocated memory.
quote:

It looks like I am giving a lecture on the CRT and its types, I am not…

i seem to be giving one
quote:

are those the errors when you don’t get a fancy shanty dialog box, instead a general MB_OK box telling you of an access violation? If not, what are you talking about?

ctrl+b, data tab. i'm not sure if you can enter a variable name in; i stick in a pointer address that i want to watch casted to some pointer type. let's say i have a variable at address 0x1234 that is trashed by something; i would set a breakpoint at (int*) 0x1234 and wait for it to be hit, then look around call stack/watch windows to see what hit it. of course, you must know the address of the variable to look at, and you must initialize the breakpoint when the value in it is still good. note that restarting your program will usually invalidate all hardcoded memory breakpoints (that is, they will point to some different memory area, and you'll have to reenter new memory addresses).
quote:

What does it mean when my heap is corrupted, and how may I have cause it.

it usually means that the supporting data structures of the heap are overwritten with junk. let's say that heap maintains a linked list of free blocks; if you've trashed the 'next' pointer in one of the nodes by writing beyond bounds of a new-allocated array, the heap manager will not be able to go through all nodes and will say that heap is corrupt.
quote:

When I compile with the multithreaded debug versions of the CRT, I get the good’ol ASSERT(_Crt…); memory heap checking bs. What might be the problem…

out of bounds array access, freeing memory twice, writing to a freed memory block, writing to a not-yet-allocated memory block (this shouldn't be the case usually), among the most common. post exact error message and whatever else the debugger is telling you. look at the call stack and variables window, as usual, to see which variables are bad. look for magic values, such as 0xcc, 0xfeee, 0xcdcd, and similar.
quote:

With the retail versions of the CRT, everything works (well… prior to this 9 day problem that I am having ).

always fix your problems in debug builds. building in release merely delays the problem until its detection is between hard and impossible, which is what you've experienced based on your original post. remember: bugs very rarely go away by themselves. get a clean run in debug build first, until you do so, forget about building in release.
quote:

What is the difference (and does it affect my code execution, like does this cause an error if I have it wrong) between a __cdecl function declaration and a __stdcall declaration. I remember having a __fastcall as my default declaration, and that worked too.

cdecl is the only calling convention out of the two supporting variable number of arguments to the calling function. stdcall's cleanup code is slightly faster when compared with cdecl on a function-by-function basis. however, these days msvc doesn't adjust esp after every function call, which means that cdecl may end up being faster than stdcall. i'd keep the default calling convention on cdecl, and make sure you always match calling conventions when you cast function pointers. calling convention mismatches are known to cause unrelated crashes and/or memory corruptions elsewhere in your code.
quote:

When I trace my code, the deadliest part of the code is when the CRT calls AllocHeap (or something like that), and one of its parameters is “__crtheap” meaning that it is allocating some memory onto the heap. Why would this cause errors if the Application and the engine DLL are both using the same CRT heap?

see above; it's because of the internal crt structures.

do you get the feeling that these things take a while to type?

[edited by - niyaw on January 22, 2003 1:08:11 AM]

##### Share on other sites
The only time I''ve had to explicitly deal with cross-process memory sharing is when I use a WindowsHook (for mouse/keyboard hooking for all applications), since my .dll gets loaded into each application, and grabs only the keys for the app it''s loaded into.

Since your .dll only has to talk to the .exe it''s loaded in, you don''t need to do anything funky to get it to work. I suspect that your bug may lie somewhere else in your system.

##### Share on other sites
One thing that''s been overlooked in the original post is that problems arise when the code is switched from a debug build to a release build.

AFAIK, with MSVC, in a debug build, local variables are initialised to zero whereas in a release build, the local variables are not initialised and contain the values that are present in the memory when they are assigned addresses (which will invariably won''t be zero), if you see what I mean.

So, for example:

void afunc (void)
{
int a;
if (a == 0) // compiles in MSVC V6.0 without warnings in release mode!
{
// true in a debug build
}
else
{
// 255 out of 256 times will be true in release
}
}

So, either initialise everything or create your own assert style macro that can be compiled into the code for both relases and debug builds and then add lots of asserts into your code. It''s good engineering practice to validate all parameters prior to using them (to make a reference to the forum name).

Skizz

##### Share on other sites
quote:
Original post by Anonymous Poster
One thing that's been overlooked in the original post is that problems arise when the code is switched from a debug build to a release build.

quote:

AFAIK, with MSVC, in a debug build, local variables are initialised to zero

not quite. they are initialized to 0xCCCCCCCC.
quote:

whereas in a release build, the local variables are not initialised and contain the values that are present in the memory when they are assigned addresses (which will invariably won't be zero), if you see what I mean.

at least you got this right.
quote:

So, for example:

void afunc (void)
{
int a;
if (a == 0) // compiles in MSVC V6.0 without warnings in release mode!

set your warning level to 4, and you'll get one. learn to use your compiler before complaining about its shortcomings.
quote:

{
// true in a debug build
}

msvc6: false in debug build, due to the reason i stated above.

msvc7, which original poster is using: not only false, but that line will cause a runtime error, "accessing an uninitialized variable", if you use the runtime checks provided by msvc. these checks should be enabled by default in debug builds.
quote:

else
{
// 255 out of 256 times will be true in release
}
}

i love %d out of %d and %.3lf-formatted statistical figures. would you mind sharing your source?

[edited by - niyaw on January 23, 2003 4:34:08 PM]

##### Share on other sites
Ok, I''ve been programming Java for far too long now. :-( And I''ve not the £1000+ spare to migrate to V7 (plus upgrading my PC to run it, £100''s more), the cost / benefit just isn''t viable.

Anyway, my example wasn''t the best I admit.

But, in my defence, the original post did say:

"I run the application in the debugger… nothing goes wrong, works like charm, 400+ FPS and all is smooth. Wait though, I am running in release mode with the debugger enabled, [cut] then BAMO! I get an error stating “Unhandled exception at 0x77f51ec7 in XYZZY.exe: 0xC0000005: Access violation reading location 0x011bb77d.”, so I break it."

There''s no mention of it going tits-up in a debug build.

But you''re right, 0xcc and not 0! Doh!

My example wasn''t the best either. Had I have put ''char a'' instead of ''int a'' then it would be a given value, statistically speaking, 1 out of 256 times. Sorry about the confusion.

I have a thing about warning level 4 from a previous MS compiler that would throw up numerous (read: pages of) warnings about ''windows.h''. But, checking it now it compiles fine. Bizarre, MS fixing something ;-)

Personally speaking, I understand my compiler very well (but, like I said above, it''s been a while and I must get re-acquainted soon). You''re obviously one of the very small minority that does too - I''ve been programming games professionally for 12 years and have yet to meet more than a couple of programmers that knows what the compiler is doing (especially the precompiled header options!). Although it can go too far, I''ve been to a presentation about this where the speaker was writing incredibly convoluted and obfuscated C code to get the compiler to generate the optimum ASM code he wanted - which was really bad engineering as the output was dependant not only on the compiler version but the compiler options used! He should have just written it in ASM to start with - it would have been much clearer.

Anyway, it wasn''t the best of examples, but the concept is sound - data is initialised in a debug build, whereas in a release build it isn''t. It applies more to dynamically allocated memory (as the compiler can''t check that the data is initialised) than to local variables. (And in this case, in debug, it''s initialised to 0xcd).

One other difference between the two builds that I''ve seen cause problems in the past is writing past either end of a dynamically allocated memory block. In debug build this won''t causes a problem, if it''s just a few bytes over, as there are guard blocks present (0xfd''s). In release, you''d be overwriting heap memory management data, thus corrupting the heap.

Skizz

##### Share on other sites
quote:
Original post by Anonymous Poster
I have a thing about warning level 4 from a previous MS compiler that would throw up numerous (read: pages of) warnings about ''windows.h''.

which is why you compile standard headers on level 3 with pragma warning push/pop, and your code on level 4. same goes for stl.
quote:

But, checking it now it compiles fine. Bizarre, MS fixing something ;-)

yes indeed.
quote:

Anyway, it wasn''t the best of examples, but the concept is sound - data is initialised in a debug build, whereas in a release build it isn''t. It applies more to dynamically allocated memory (as the compiler can''t check that the data is initialised) than to local variables. (And in this case, in debug, it''s initialised to 0xcd).

i think you''re (still) implying that debug builds hide bugs that show up in release builds. correct me if i''m wrong. but if i''m right, it''s plainly not so. debug capabilities of msvc are incredible for finding and fixing bugs. granted, they won''t do much for a programmer who "debugs" with print statements or log files, but those magic value often tell you exactly what the problem is: accessing uninitialized memory, writing past allocated memory, freeing memory twice, accessing freed memory. msvc''s debug heap detects underwrites and overwrites, double-frees, memory leaks, memory states and state differences, breaking on allocation id, and custom alloc hooks. debugging broken debug heap in msvc is so ridiculously easy that it''s not even funny, compared to let''s say tracking down broken kernel heap (exception in kernel32: bad heap pointer passed from shell32, that''s five functions away from your code or there are no your symbols in the call stack, now what?). msvc7''s runtime checks are even better: detection of buffer overflows (never had this occur on me yet, so can''t comment), catching uninitialized variables, and so on.

of course, warning level 4 is mandatory. why spend time debugging when your compiler can tell you *exactly* what the problem is? level 4 catches assignments in if statemtents and reads of uninitialized data, among other things.
quote:

One other difference between the two builds that I''ve seen cause problems in the past is writing past either end of a dynamically allocated memory block. In debug build this won''t causes a problem, if it''s just a few bytes over, as there are guard blocks present (0xfd''s). In release, you''d be overwriting heap memory management data, thus corrupting the heap.

again, in debug build you''ll get an assertion from debug heap telling you that you wrote past allocated memory. not only that, you''ll be given allocation number, on which you can break next time you run the program. one minute later, you know exactly which object or array writes past itself.

there''s no way you can blame msvc debug options for hindering your ability to debug programs. it''s just the opposite.

##### Share on other sites
AP:
quote:
Original (second) post by AfTeRmAtH
I am having these sort of crashes in both debug and retail builds.

that''s where you say "what the hell are you doing in the release build if your debug build is broken?"

##### Share on other sites
These new debugging tools in V7 sound good. Still doesn''t quite make forking out that much dough worth it since I''ve written a memory manager that does a lot of what you describe and is platform independent (consoles don''t have anything like the tools the PC has :-( ). PC programmers have it way too easy. ;-) And don''t get me started on the tools for mobile phones.

To clarify, in V6 doing:
1 int *a = new int [10];
2 printf ("a [20] = 0x%08x\n", a [20]);
3 a [20] = 0;
4 printf ("a [20] = 0x%08x\n", a [20]);
5 delete [] a;
is how bugs can hide in debug builds. With V6, this code doesn''t generate any assertions at all in debug, and runs without error, but who knows what will happen, in release.

Actually, this example runs fine as is, but I hope you can see what I''m getting at - you could be overwriting harmless data in a debug build, but critical data in release build since the amount of heap management data is different for each version (which isn''t the case in my cross-platform manager, for this reason). So, doing:
int *a = new int [10];
printf ("0x%08x\n", a);
int *b = new int [10];
printf ("0x%08x\n", b);
in debug produces:
0x00301cc0
0x00301c60
and in release:
0x003002f0
0x003002c0
[note - not run from within a debugger], so writing to ''b [0x30 / sizeof (int)]'' doesn''t affect anything in debug but overwrites the start of ''a'' in release - a bug in release that runs fine in debug.

Zapping the guard block will produce an assertion on line 5 though (but the guard block is only 4 bytes long).

I must stress that this has all been done on MSVC V6 and I know that the original poster is using V7 (.NET), it would be interesting to see if the new debugging tools cope with this type of problem.

Skizz

##### Share on other sites
most more or less experienced programmers won't just access 20th element of a ten-element array. granted, msvc isn't godlike, and won't help with all errors. however, it will help with the (more?) common and subtle off-by-one errors, and contiguous memory overwrites.

now that i think about it, if something manages to slip past debug diagnostics, it can hide easier in debug than in release. for instance:

- corrupted stack, i.e. from calling convention mismatch: should be caught by the stack validator that's run after every function call. if not caught, problem will likely never surface in debug because every function saves and restores esp.* in release, all hell may break loose if esp isn't restored for a couple functions up the stack. small functions with few or no local variables are likely to not have a stack frame, making them prime candidates for passing havok on.

* a famous example of this particular scenario is c-style casting of mfc message handlers with improper signatures. always works in debug and never in release.

- optimized code, string pooling, removal of unused code, etc: more chances of overwriting something that's going to run; more disastrous consequences of running junk.

so you were right, too.

[edited by - niyaw on January 24, 2003 8:10:44 AM]

##### Share on other sites
It''s imperative that you test code in both release and debug builds on a regular basis for these reasons.

##### Share on other sites
Wow, thank you all for the replies! I notice that this is a very “professional” discussion where each person writes a lot, heh. I usual think that if a person takes the time to write so much, then they are serous! So thanks niyaw ! (he looks like he knows his stuff everyone!).

I want to make it clear that the application runs without any errors at all (in debug and release mode), only when the debugger is active… meaning when I execute it within the MSVC .NET IDE.

I took the advice and ditched the “retail” CRT libraries (and switched to the “debug” ones)… and it actually is getting me somewhere. I think that I am going to pack all my stuff together, and (as he insisted) send my code to niyaw ’s way, maybe you can look at it (and the compiler options and whatnot), and find what’s wrong (I don’t know, but that seems wrong that you are willing to take all that time, but ok. I appreciate it man).

First things first:
the heap is fooked!
I don’t know how, but somewhere in my code I am messing something up.
These are the things that I did so far¹:
• At the start of the “execution function”², I created a thread that has an infinite “while” loop (well it checks if a Boolean is set, just so I can make it false during debugging) calling the _Crt memory checking function thingy. It looked something like this:
while(bThreadRunning){_EASSERT(...);}

That was a big mistake.
• Called:
 _CrtSetDbgFlag(_CrtSetDbgFlag(_CRTDBG_REPORT_FLAG)|_CRTDBG_CHECK_ALWAYS_DF);

So that memory checking will be executed on every call to the (debug) “malloc” function. This lead me to some code that was making an STL “deque” structure push an element to its back… this got me thinking, is STL safe when doing this?

¹I am now talking about a whole different project, I am now not executing the game, but I am now executing an application that calls a function named “TestRun” from the engine DLL. This is prety self-explanatory… this is just a way for be to test my engine and have all the testing code within the engine DLL. I don’t think that this has to do anything wit the problem (that code is still being evoked by an EXE within a DLL), but I thought id let ‘you’ know, maybe it does make a difference, you tell me.
²As stated above, this function is merely one of the hundreds found inside the engine DLL.

So this obviously means that my heap was broken somewhere.
I know that it is not within any of the engines “internal” code, because I compiled a pong game that uses the engine using all the same (debug multithreaded CRT) and nothing came up (no errors).
I know for sure that the error is because I am accessing and passing data to the engine functions, more precisely when I set an animation sequence (to the 2d sprite animation container) that I got from the container (well ok “class”, but “container” sounds more “cool”).

So before I go get that code ready to give to niyaw , I am going to say something about what people said.

quote:
Magmai Kai Holmlor
It's imperative that you test code in both release and debug builds on a regular basis for these reasons.

Yes I know, but my code does work in release and debug mode… just not with the debug CRT (man does that sound stupid or what?). I guess you are right… this sort of makes me a bad programmer… shame on me!

*** some time later ***
okay I am really getting somewhere, thanks to everyone who helped (especially niyaw ), I'll post here if I have any more questions

[edited by - aftermath on January 25, 2003 8:42:04 PM]

##### Share on other sites
That can mean one of two things. You said you are using a dll. Both the project and dll need to use the same version of the CRT, or the code is broken in both cases.

The debug version detects problems whereas the release doesn't. So just because the release version does not report a problem, does not mean it's not occuring. (For that matter, a debug build won't catch all problems either). If you are allocating memory (or objects) in the dll and destroying them in the application, that can cause problems too - xsp if they both are not using the same CRT flavor.

Nominally, you can provide a different library for each configuration (all six).

At the least, provide a debug and release flavor, MyLibraryD.dll & MyLibrary.dll. Then you need to link to the appro. one for your build.

First, make certain all the sub-projects are using the same debug version of the CRT. If the problem still occurs, begin commenting out sections of functionality until the problem goes away. Add stuff back in until it comes back. Rinse-wash-repeat, until you narrow down the cause.

Also, if you are receiving a heap assertion, there's tons of information there about what happened, and who-done-it. For instance, if it's a leak, it will tell you the allocation # and you can configure the CRT to break on that allocation using
CrtSetBreakAlloc.

CRT Debugging

An assertion failure means the code has detected an invalid state. You have done something that you are never suppose to do.

[edited by - Magmai Kai Holmlor on January 27, 2003 4:10:19 PM]

##### Share on other sites
What huh? Psh! I am so over that now!

Hehe, thanks everyone… without all you yelling at me what I am doing in release mode when debug mode doesn’t work really turned on a bit In my brain. Now everything works flawlessly (and faster) in bother builds THANKS!

##### Share on other sites
what was the error?

##### Share on other sites
There was two errors in my code actually! And I was quite foolish to not notice them before, because it was one of those things that are just stupid… you know? I hate stupid errors!
Yeh, the first error was as follows:
All my animations are stored in an STL list and where, as everybody knows, the first element is element 0; meaning that if there is 0 elements, it means that there is actually one.
I needed to translate an STL list (of the animations sequences) into a linier array. So I had the list::size() function as the first parameter for the ‘new’ operator. There was one sequence, and that made the new operator work like it was suppose to return 0 memory, witch it did. Big mistake buddy.
So I just added a “+1” and everything works fine.

The second mistake was just like the first, only it was working with chars instead of animations sequence.

Thanks again everybody… I actually (no I mean really, this helped) would still be stuck if it wasn’t for you.

dynamic sigs are fresh!