Sign in to follow this  

What heap wil dynamic memory be created on

This topic is 1113 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello all.

 

I am busy working on makeing a vertual file system, And I've run into a problem that I may have been cruising around because all my DLL libraries are using the same platform toolset( And that means the same heap right? ).

That problem is: Who is responsible for deallocating the memory: Caller or Callee ?

 

So if I create a structure like so:

struct Blob
{
          unsigned char* pData;// std::vector< unsigned char > Data
          size_t Length;
};

(Would the vector dealoccate itself from the right heap?)

 

As far as I know if the structure was allocated on heap A and the caller is using Heap B and deallocates it there will be a crash or segfault or something, Right?

 

So the bottom line is: How do I make sure that the memory is deallocated from the right heap.

 

 

Am I correct in my understanding of dynamic memory here?

 

Many thanks

Ryan.

Share this post


Link to post
Share on other sites

On Windows, HeapAlloc() and HeapFree() can be used instead of the runtime specific memory management functions (new/delete, malloc/free) to ensure that dynamically allocated memory is always freed from the correct heap. Using runtime specific versions, smart pointers and whatnot internally within a module/DLL is fine but if you pass these memory blobs out of the DLL to be freed elsewhere, you should consider HeapAlloc/HeapFree as a safe, although a bit cumbersome, option.

Share this post


Link to post
Share on other sites

On Windows, HeapAlloc() and HeapFree() can be used instead of the runtime specific memory management functions (new/delete, malloc/free) to ensure that dynamically allocated memory is always freed from the correct heap. Using runtime specific versions, smart pointers and whatnot internally within a module/DLL is fine but if you pass these memory blobs out of the DLL to be freed elsewhere, you should consider HeapAlloc/HeapFree as a safe, although a bit cumbersome, option.

Can you give an example of how they are used, I google'd but could not find an example.

Perhaps I am just a bit daft right now dry.png?spov=1416834173321

Or is it just a simple matter of using those to create the buffer needed for the file ?

 

 

 

Am I correct in my understanding of dynamic memory here?

No.
There is only 1 heap.
The reason some libraries can’t deallocate allocations made by other libraries is because each library puts meta-data before an allocation to tell it how much has been allocated and etc. The format of this data may vary from library-to-library, and since it is necessary to deallocate it will cause a problem if an incompatible library tries to deallocate memory allocated by another library.

So each platform toolset option(v120_xp, v120, Etc - Visual Studio) would use its own format for the meta-data ?

 

 

Who is responsible for deallocating the memory: Caller or Callee ?

The caller. It is your responsibility to make sure that the function you call to deallocate is designed to pair with the function you called to allocate.

If you use new, free with delete.
If you use malloc(), free with free().
If you use a custom allocator, use the same custom deallocator.

It is that simple.


L. Spiro

 

So if a provide a method/function for allocation, it would be placed on whatever heap the DLL is using ?

 

If so: It would be a simple matter of two options:

  1. Have the caller call the method with a certain parameter set to a predefined value->return the needed size of buffer->call again with user allocated buffer.
  2. Have a structure that allocates/deallocates the buffer.

Share this post


Link to post
Share on other sites
No. There is only 1 heap.
The reason some libraries can’t deallocate allocations made by other libraries is because each library puts meta-data before an allocation to tell it how much has been allocated and etc.

That's correct for Linux/Unix. For Windows (which this seems to be about), as a blanket statement, this is wrong (although the part about metadata is correct).

 

DLLs do not necessarily have their own heap, but they are allowed to create one, and some indeed do. This creates a "private" heap which reserves a contiguous area of address space (explained in the Remarks section here) with a handle different from the one returned by GetProcessHeap.

 

One notable example is Microsoft's CRT (and hence any library that dynamically links against it), which will lead to problems if you mix statically linked executables with dynamically linked DLLs, as outlined for example here. The bottom line is that the linker is smart enough not to let you mix the two CRTs in the executable, but it is possible (and in principle legitmate, and undetectable for the linker) to have some other DLL linked to a different CRT than the main program, and boom.

 

Other DLLs may create private heaps because the default heap is the "classic" one, not the "low fragmentation" version on Windows versions up to 7 (I may be lying here, it might be Win7 inclusive, not sure...). For some libraries, it may make sense to request the low fragmentation heap.

SQLite is one example of a library that calls HeapCreate.

 

In addition to that, every compiler (that includes different versions of the same compiler that use a different ABI, such as e.g. GCC/MingW 4.7 ?  4.8) adds some metadata to allocations. When you write something like new or delete in your program, this usually maps to malloc and free, plus calling constructors and destructors (the C++ standard does not say that, so this is in principle not correct, also you could overload operators, but it is correct in practice).

malloc and free, on their part, do some voodoo (usually something like add 4 bytes for the allocation length, and then round up to 8-byte alignment, which is the "real" address they'll return and accept), and call HeapAlloc and HeapFree. Whatever they do is not specified and may vary among compilers.

 

So, in short, you can't expect that deleting something will work reliably (it might work, accidentially!) if the whole combination of compiler plus module is not 100% identical in respect to how the object was allocated.

 

 

 

So if I create a structure like so:

struct Blob
{
unsigned char* pData;// std::vector< unsigned char > Data
size_t Length;
};

(Would the vector dealoccate itself from the right heap?)

The vector will, depending on the situation, call the standard allocator's deallocate function or operator delete, which may in fact do nothing (it might put the memory block on a free list) or which may eventually -- possibly via a roundtrip through free()-- call HeapFree. It will not magically figure out the correct heap, if there are several ones.

 

By the way, I'm not quite sure what the intent is with that raw pointer, the length, and the vector. In any case, the vector would need to have a "life" in some other place, or the raw pointer could not possibly be valid (if you do something that causes the vector to resize, it won't stay valid either). There's probably an easier and safer way of doing the same thing, such as  using blob = std::vector<unsigned char>; -- vector already knows its length (size) and data() is always a valid pointer.

Edited by samoth

Share this post


Link to post
Share on other sites
You can make the operations on your data structure use the right heap heap by giving your class constructors and a destructor and all vector operations wrapped in member functions and implement those in the DLL (in a .cpp file compiled into the DLL) instead of a header (you could also write a custom allocator that's implemented in the DLL instead of wrapping all the vector operations). e.g.

// foo.h
class foo {
  using container = std::vector<int>;
  container data;

public:
  foo();
  foo(foo const&);
  foo& operator=(foo const&);
  ~foo();


  using iterator = container::iterator;
  using const_iterator = container::iterator;
  using size_type = container::size_type;

  void push_back(int);
  iterator erase(iterator);
  iterator begin();
  const_iterator begin() const;
  iterator end();
  const_iterator end() const;
  size_type size() const;
  bool empty() const;
  // etc.
};
// foo.cpp
#include "foo.h"

foo::foo() = default;
foo::foo(foo const&) = default;
foo& foo::operator=(foo const&) = default;
foo::~foo() = default;

void foo::push_back(int value) { data.push_back(value); }
foo::iterator foo::erase(iterator loc) { return data.erase(loc); }
// etc.
Because all the allocations related to foo's data member happen inside foo.cpp.

If your vector is of a custom type and not just int or unsigned char or whatever, you could also just use an extern template for the vector.

// foo.h
struct foo_data {
  int data;
};

using foo_vector = std::vector<foo_data>;
extern template class __declspec(export) foo_vector;

struct foo { ...
// foo.cpp
#include "foo.h"
foo_vector; // instantiate the code for vector<foo_data> in this .cpp file
I haven't ever actually the tried the latter, but it's documented by Microsoft more completely at http://support.microsoft.com/kb/168958.

In general, though, you should try to just stick to simple C wrapper APIs for DLL boundaries. You'll save yourself a huge ton of pain.

Share this post


Link to post
Share on other sites

This topic is 1113 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this