Personally, I always use pointers in container templates. This avoid unnecessary memory copies and constructor/destructor calls.
I'm a bit of a memory nazi though, so I prefer doing the new and delete myself.
Unfortunately your paranoia over copying means that you are pretty much dropping performance all over the place as between the de-referencing and the data misses you are giving the CPU a hard time.
Memory is SLOW, if you can you ALWAYS want to keep things as close together as you can when processing to take advantage of pre-fetching and caching of data. Stalling out the CPU for many cycles while it wanders off to find the data you've just asked for is not a productive use of anyone's time.
Edit: Part of me thinks that everyone should be made to program on a system with limited cache and an in-order arch. just so they can appreciate how many sins an OoO processor covers up...