Strange memory allocation bug with new[]

Started by
8 comments, last by tivolo 10 years, 3 months ago

Hi

I have ran into a strange problem..

I have this piece of code:


template <class T>
void my_vect<T>::myAlloc(size_t dim)
{
	try
	{
		dat=new T[dim];
	}
	catch(bad_alloc dat)
	{
		err::MyErrorHandler.Error(ALLOC_ERROR);
	}
}
 
Later "dat" is causing the heap corruption, even if it shouldn't. After many many many hours of figuring out why, I found out that the value of my "dat" pointer isn't the same as the value returned by the new[] operator (found it out by stepping into the new operator and looking what it returns). For example new returns 0x...e18, but "dat" is 0x...e1c. The offset is always 4 bytes long. This causes my program to fail when using for example _msize() function.
Why is it happening?
Advertisement

msize is a C-api made to work with malloc, not with new, that might be your problem.

It is also microsoft only, so if you want to write portable code, you should avoid using it at all.

You have what is presumably a member variable called "dat," and you also then shadow this variable with a local variable named "dat" inside your catch block. While that might not be related to the problem you are having (at which we can only guess, because memory corruption bugs usually require more code than this to properly diagnose since they could in theory happen just about anywhere), it's not a good habit to get into.

The operators new, delete, new[] and delete[] have two purposes: 1) to allocate memory, and 2) to construct and destruct objects. Operator new[] constructs an array of objects after allocating the memory for those objects. When delete[] is called, it must call the destructor on each of the objects in the array, but delete[] doesn't take an array length parameter so it has no way of knowing how many objects are in the array. So delete[] must figure out, somehow, how many objects it needs to destruct. The C++ standard doesn't specify how delete[] should figure this out (as far as I know) but a common implementation is for new[] to put an int at the front of the memory allocated that contains the length of the array.

So, that's what you're seeing. The new[] operator is putting the array length at 0x..e18, and the actual objects you allocated start at 0x..e1c. When delete[] is called, it'll take the pointer passed in, subtract 4 to get the pointer to the array length, destruct the number of objects specified in that int, and then free the memory.

As Olof Hedman says, msize is a C function, and it has no idea that any of this array trickery is happening, so it doesn't have any chance to succeed. You MIGHT get msize to work by calling it on ptr-4, but even if that did work it wouldn't be portable.

msize is only defined to return anything sensible for blocks allocated by malloc, calloc or realloc, calling it with any other pointer is undefined behaviour.

It might seem to work, for some compiler version, but you should never rely on undefined behaviour.

if you are "lucky", then new is implemented using malloc, and then it would work, but its still undefined behaviour, and could change at any point.

You need to save the size to a separate member if you need it later (or use std::vector that already implement this)

Thank you for replies, especially Samith. But I have checked it and when I created other project, and then created an array of objects using new, the returned value was the same as the pointer value.

Olof Hedman - I also tried to overload the new operator, so it returns malloc(), but nothing changed.

The problem is in the allocation itself, because if I do myAlloc(), and right after it _msize(), I get a heap corruption error.


Olof Hedman - I also tried to overload the new operator, so it returns malloc(), but nothing changed.

Did you also overload operator new[]?

I don't have extensive experience in the area but I remember reading that operator new[] is allowed to (and does on the common compilers) to store some information of its own in the allocated memory (typically the number of instances). That behavior cannot be changed by overloading the operator.

That aside, I would advise against using _msize. A cursory research suggests it is MSVC-only and if there is even the remotest chance you might have to compile the code on a different compiler anywhere in the future something like that is just not worth it.

As long as you send the exact pointer that was returned from malloc to msize, it should work.

Also malloc will store some housekeeping information in the memory it allocates, usually stored just before the pointer that is returned.

If this information is corrupt, or isn't there, like for example if the memory was not allocated by malloc, then msize will fail and tell you the heap is corrupt. (though it isn't really corrupt, its just not memory from the heap it expects its from)

The operators new, delete, new[] and delete[] have two purposes: 1) to allocate memory, and 2) to construct and destruct objects. Operator new[] constructs an array of objects after allocating the memory for those objects. When delete[] is called, it must call the destructor on each of the objects in the array, but delete[] doesn't take an array length parameter so it has no way of knowing how many objects are in the array. So delete[] must figure out, somehow, how many objects it needs to destruct. The C++ standard doesn't specify how delete[] should figure this out (as far as I know) but a common implementation is for new[] to put an int at the front of the memory allocated that contains the length of the array.

So, that's what you're seeing. The new[] operator is putting the array length at 0x..e18, and the actual objects you allocated start at 0x..e1c. When delete[] is called, it'll take the pointer passed in, subtract 4 to get the pointer to the array length, destruct the number of objects specified in that int, and then free the memory.

As Olof Hedman says, msize is a C function, and it has no idea that any of this array trickery is happening, so it doesn't have any chance to succeed. You MIGHT get msize to work by calling it on ptr-4, but even if that did work it wouldn't be portable.

Solid advice, here's a bit more detailed info: http://molecularmusings.wordpress.com/2011/07/05/memory-system-part-1/

As Samith and others have said: never mix C and C++ memory functions! Don't mix new & malloc, delete & free, or any other C function with new/delete. It will not work, as soon as you start using non-PODs (it mostly works for PODs because delete[] then doesn't allocate extra bytes).

This topic is closed to new replies.

Advertisement