There's nothing wrong with your raw pointer data structure -- now that we have a good, functional set of smart pointers you should start thinking of raw pointers as non-owning pointers. As long as you never need to take (even temporary) ownership (that is to say, lock the pointed-to thing for some time, so that it doesn't get deconstructed before you're done with it) this is just fine.
Now, if the real problem you're getting at is that you *do* actually need to take temporary ownership from that second data structure, then you do need something like shared_ptr because you really do have shared ownership -- or alternatively, depending on the circumstance, it could make a copy of the thing.
Then you have some options -- your second data structure could contain shared_ptr if you need for an item's presence in the second structure to be a guarantee that its available -- but you'll have to remove it from both data structures (and destroy all other shared_ptrs) before what it points to is destroyed. Or, your second data structure could contain weak_ptrs, which are essentially an *option* to generate a shared_ptr to the thing -- if the shared count has gone to zero and then you request a shared_ptr to it through a weak_ptr that you still have, you get an empty shared_ptr back.
Weak_ptr works by maintaining a second reference counter in the shared_ptr control block (a weak ptr is just a pointer to the control block). When no more shared_ptrs point to the thing, the shared count goes to 0 and the thing is deleted, but the control block itself remains as until the weak count also goes to 0 (the control block only goes away when both the shared and weak counters go to 0). This is what allows a weak pointer to know whether the pointed-to thing is still there or not, and is also the reason why you have to ask it for a shared_ptr, and can't get to the pointed-to thing directly through a weak_ptr.
Now, the final consideration of using shared_ptr along with weak_ptr is whether you want to create the initial shared_ptr with std::make_shared, because if the pointed-to thing is small or very small, the shared_ptr control block will actually contain the memory for the pointed to thing inside itself, and because of that, the memory for the thing will not be reclaimed when the shared count goes to 0, even though the things destructor will get called. 98% of the time, this design is a good thing, and it won't do this for large things. Very small things, a few machine words, might fit inside the alignment padding or inside of a cache line, or whatever -- they're essentially "free", but sometimes the small-but-not-very-small things can add up -- this sort of thing, for me, falls into the "don't worry about it before you need, but keep it in mind" class of problems. The problem also is a non-issue if you have good discipline over those weak_ptrs -- you can't always say it, but often times a proliferation of weak_ptrs is indicative of a design smell, or of a program bug that leaks objects still holding onto a weak_ptr.
