I just upgraded from 32bit Windows to x64... Now a program I run uses realloc() and once the integer value of the address reaches greater than ~2000000000, it crashes - I suspect there's some gcc argument which will prevent this... any ideas?
32bit/64bit Woes?
Could be various issues.
One of the more likely causes is pointer truncation. It is a relatively common problem from header mismatch where something is referencing the old 32-bit headers rather than the 64-bit header, or a link issue or code duplication issue where something is using the 32-bit version of the function.
A little bit of time with the debugger and the disassembly window can show you if the pointer is getting truncated. Hopefully it would be as simple as using a watch window where a pointer suddenly changes from an extended 64-bit address to a 32 bit address. Stepping through the code with a mixed disassembly enabled can also help reveal the mismatch, such as using the 32-bit ecx instead of the 64-bit rcx register.
One of the more likely causes is pointer truncation. It is a relatively common problem from header mismatch where something is referencing the old 32-bit headers rather than the 64-bit header, or a link issue or code duplication issue where something is using the 32-bit version of the function.
A little bit of time with the debugger and the disassembly window can show you if the pointer is getting truncated. Hopefully it would be as simple as using a watch window where a pointer suddenly changes from an extended 64-bit address to a 32 bit address. Stepping through the code with a mixed disassembly enabled can also help reveal the mismatch, such as using the 32-bit ecx instead of the 64-bit rcx register.
but the 32bit exe should run on 64bit through WoW64
Yes, but that has nothing to do with what frob said.
What exactly should i do
Debug your app. Easily said that done, I know, but you already know when and where the crash happens. You can break right before it crashes, switch to disassembley view, and check what happens.
BTW, what exception are you getting?
Author
the problem is realloc is returning NULL after the integer pointer address size goes above ~2billion. Not sure why it goes so high. it either grows "faster than it should" (program only uses about 400MB) - or it "jumps" from some low point to some high point
it either grows "faster than it should" (program only uses about 400MB) - or it "jumps" from some low point to some high point
realloc works fine. The reason it uses more memory than you allocated is because of fragmentation. When you allocate a buffer, the allocation has to be contigous, so depending on your allocation pattern, you'll see some overhead ('some' can become 'extreme' sometimes).
Also, you can't really deduce how much memory your application consumed based on the address, as allocation doesn't start from 0. The task-manager will tell you exactly how much memory your app actually consumes.
Anyway, based on your description, I suspect you have a memory leak.
Author
yes, task manager reports 400MB. this exact same code ran fine on win32.. no memory leaks
No it does not.
Using realloc is the problem. If you constantly increase an array by a small amount (say 1), there is a good chance that the array can not be increased because s.th. else was allocated behind it. This results in realloc allocating a completely new chunk of memory somewhere else and copying all the contents over. The prior location is then free for other use. However, since you keep increasing your array size, it will never again fit into that gap. Even worse, in some (not uncommon) situations, this happens with every resize/realloc of the array. You end up with a crap ton of gaps that are basically "free" but can't be used. This is the fragmentation that satanir meant.
Using realloc in this fashion can easily result in your application requesting multiple GBs of memory from the OS although you only need a few MBs or KBs. And at some point, your virtual address space simply overflows. That is what you are seeing.
Using realloc is the problem. If you constantly increase an array by a small amount (say 1), there is a good chance that the array can not be increased because s.th. else was allocated behind it. This results in realloc allocating a completely new chunk of memory somewhere else and copying all the contents over. The prior location is then free for other use. However, since you keep increasing your array size, it will never again fit into that gap. Even worse, in some (not uncommon) situations, this happens with every resize/realloc of the array. You end up with a crap ton of gaps that are basically "free" but can't be used. This is the fragmentation that satanir meant.
Using realloc in this fashion can easily result in your application requesting multiple GBs of memory from the OS although you only need a few MBs or KBs. And at some point, your virtual address space simply overflows. That is what you are seeing.
Well if is program is a 32 bit executable, here's your problem. Even if your running on a 64 bits OS, you just can't allocate more than 2 gb of memory. Even with the LARGE ADDRESS AWARE thing, i think you'll be limited to 3gb instead. If you need that much memory, why not compile a 64 bits exec??
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement