Passing shared_ptr between modules

Started by
8 comments, last by Jan2go 9 years, 10 months ago

I have a singleton class (EventManager) in a static lib that gets linked by both the .exe and a dynmic lib. Events are passed around as std::shared_ptr<IEvent>. The EventManager stores all queued events in a std::list and pops them out after handling them. This works fine as long as all events were added from the exe. If I add events from inside the DLL, pop_front throws an exception at the last event that was added from inside the DLL. It doesn't matter whether the DLL adds one event or 1000, only the one that was added last throws the exception.

This is the exception:


Debug Assertion Failed!

Program: ...ts\visual studio 2013\Projects\QueueTest\Debug\DynamicLib.dll
File: f:\dd\vctools\crt_bld\self_x86\crt\src\dbgheap.c
Line: 1424

Expression: _pFirstBlock == pHead

I'm using the Visual Studio 2012 with compiler for this.

I know that you should not deallocate memory in any module other than the one that allocate the memory. However, from what I've read std::shared_ptr can be used to avoid this problem as it contains a pointer to the deleter for the memory.

I attached a sample project with this problem to this post.

Any ideas on how I can get the EventManager to work with events coming from a DLL?

Advertisement

If I recall correctly, the default deleter used by shared_ptr is just going to call standard delete. It will not call the delete that resides in the same module as the new that created the object. It will call the delete from the module in which the destructor is being compiled. And if your dll and your exe both have their own memory pools that they manage independently, this is where you'll encounter problems. In Visual Studio, if you compile with the option /MT (multi-threaded static runtime library), this is the situation you'll have.

There is a way to make the dll and exe share the same delete, however, and that is to use the shared runtime. This runtime is itself a dll, and both your exe and dll can dynamically link to a single instance of this dll, which means that news and deletes can interoperate between the exe and dll (or between two dlls).

To enable this, you'll need to use the /MD compiler flag, instead of /MT. I can't remember the exact settings panel it is in (VS isn't installed on my machine right now, going off of pictures from Google Images), but it looks to be C/C++ / Code Generation / Runtime Library / Multi-threaded DLL (/MD).

In some cases, it can ultimately be the right thing to create a solution where the shared_ptr (or something similar) can properly deallocate from anywhere, by storing at allocation time a stateful deallocator. Or alternatively to design a policy that avoids the issue altogether (e.g., ownership of a resource is never passed across module boundaries, only a non-owning reference). But in the basic case, using a shared runtime dll is easiest and perfectly fine. I'd recommend only going the more complicated route when you know precisely the reason in your particular case for doing so.

"We should have a great fewer disputes in the world if words were taken for what they are, the signs of our ideas only, and not for things themselves." - John Locke

However, from what I've read std::shared_ptr can be used to avoid this problem as it contains a pointer to the deleter for the memory.


It _can_ contain a deleter, but by default it just calls operator delete unless you override the deleter.

http://stackoverflow.com/questions/12340810/using-custom-deleter-with-stdshared-ptr

Are you sure you need a shared_ptr? You're almost always better off with a unique_ptr unless you really do need shared ownership. An event queue is a perfect place to consider using a unique_ptr. Someone creates it, hands it off to the queue (moving ownership to the queue), and then someone else retrieves the next event (moving the ownership out of the queue into the receiver). There is only one concrete owner at a time.

For allocation/deallocation, consider an approach where a single authority is responsible for all of this. For instance, instead of using make_unique to allocate a new event, call into your event queue to allocate a new event. The deleter of the event should likewise call the event queue's destroy event method. Then whichever module contains the event queue itself is doing all allocation and deallocation of events.

Sean Middleditch – Game Systems Engineer – Join my team!

Agree with Sean. I probably undersold the idea of designing a clear ownership policy. Even if you're not dealing with cross-module issues, it's a good practice to always spend some time thinking about your program's ownership policy. A well understood and well chosen policy can make a whole host of problems go away, or not even appear to begin with.

"We should have a great fewer disputes in the world if words were taken for what they are, the signs of our ideas only, and not for things themselves." - John Locke
According to the debugger all function pointers inside the shared_ptr go to the DLL, including those that are related to deleting stuff (at least by name). Also, all but the last event get destroyed correctly inside the DLL so I think that the deleter is set correctly.

Using the /MD flag works, however I would prefer a solution that works with /MT. I tried to switch to /MD a while back and for some reason (can't remember why, I think it was a library that I'm using) didn't get the project to work with /MD.

About the unique_ptr, the compiler doesn't let me declare a std::list of unique_ptrs in the EventManager class.

About the unique_ptr, the compiler doesn't let me declare a std::list of unique_ptrs in the EventManager class.


That somewhat surprises me. Posting the errors may be helpful; it's definitely legal to have a unique_ptr in a list. http://goo.gl/6bl6Fi

That said, std::list is Satan's data structure; consider using a deque or even a vector instead.

Sean Middleditch – Game Systems Engineer – Join my team!

This is just speculation, but it is possible that most of the events appear to get deleted fine due to implementation defined behavior of the runtime's allocator. For example, if every piece of allocated memory is part of a larger block that is predictably aligned, then on delete, the deallocator usually only needs to jump to the beginning of that memory's aligned block and fiddle with some book keeping information. This will work just fine even when technically using the wrong allocator, as long as the code is the same. But as soon as the deallocator notices that it freed the last piece of memory in a block, it might want to clean up the whole block. And to do this, it might need to check some other memory that is unique to that particular allocator (such as a list of memory blocks that the allocator is managing). If this block of memory wasn't actually allocated by the allocator currently doing the deallocation, things get confused really fast.

This might be the kind of thing that is happening when you delete the very last event. All deletes might be wrong, but the debug runtime only notices that fact on the last delete, when it tries to clean up a block.

Also keep in mind that your events are not the only thing being allocated and deallocated across dll boundaries. Every call to events.push_back() is going to use the default list allocator to allocate a linked list node, and every call to events.pop_front() will use that allocator to deallocate the node. But this is almost definitely a stateless static allocator. Since you statically link your event manager to both your exe and dll, the allocator used is going to depend on which module the calls to Queue() and HandleEvents() comes from. Calling Queue() from the dll is going to allocate from the dll, and calling HandleEvents() from the exe is going to deallocate from the exe. No amount of shared_ptr intelligence will protect against this, because this is completely independent of the memory allocated for events. Stateful allocators would solve this (at the cost of a little overhead), but have only been possible since C++11, and I'm not sure what VS2012's support of these is like.

Without stateful allocators, you pretty much need to figure out a way to guarantee that all manipulations to a container happen within a single module. In your case, you might actually try making your event manager library a dll also, instead of a static library. That way, all manipulations to the events list is guaranteed to happen within that dll, regardless of where the call to Queue() and HandleEvents() comes from.

"We should have a great fewer disputes in the world if words were taken for what they are, the signs of our ideas only, and not for things themselves." - John Locke

That somewhat surprises me. Posting the errors may be helpful; it's definitely legal to have a unique_ptr in a list. http://goo.gl/6bl6Fi

That said, std::list is Satan's data structure; consider using a deque or even a vector instead.

This is the error message. It was caused by a call to EventManager's copy constructor, so no actual problem with list<unique_ptr>. smile.png


Error 1 error C2248: 'std::unique_ptr<_Ty>::unique_ptr' : cannot access private member declared in class 'std::unique_ptr<_Ty>' c:\program files (x86)\microsoft visual studio 11.0\vc\include\xmemory0 617 1 StaticLib


Also keep in mind that your events are not the only thing being allocated and deallocated across dll boundaries. Every call to events.push_back() is going to use the default list allocator to allocate a linked list node, and every call to events.pop_front() will use that allocator to deallocate the node.

To be honest, I didn't took that into consideration until now. I tried it with a std::vector (small enough to avoid reallocation from within the DLL) and deleting object from the vector (thereby destroying them) works without problems. However a heap corruption occurs when the program exists, so there is still something that gets broken in the process.

I guess I'll try the /MD switch again, hopefully I can get it to work this time. biggrin.png

This is the error message. It was caused by a call to EventManager's copy constructor, so no actual problem with list.
Error 1 error C2248: 'std::unique_ptr<_Ty>::unique_ptr' : cannot access private member declared in class 'std::unique_ptr<_Ty>' c:\program files (x86)\microsoft visual studio 11.0\vc\include\xmemory0 617 1 StaticLib


Ah, yeah. The lack of a copy constructor in unique_ptr should cause list to not have a copy constructor which should cause EventManager to not have a copy constructor and everything just work until you actually try to explicitly copy EventManager. Older compilers lacked this feature, including Visual Studio 2012 (which you appear to be using). You might consider upgrading to 2013.

Sean Middleditch – Game Systems Engineer – Join my team!

Older compilers lacked this feature, including Visual Studio 2012 (which you appear to be using). You might consider upgrading to 2013.


I wish that would be the only feature that is not supported in VS 2012. biggrin.png From what I've read there will most likely be a new version of VS later this year; I'll probably update the project to the new compiler then (hopefully removing lots of #ifdef _MSC_VER >= 1700 smile.png).

This topic is closed to new replies.

Advertisement