A small background story may be in line here - I am a self taught C++ developer, and also a self taught graphics programmer. In general, I am a voracious reader and so when I was starting out in development, I picked up many of the habits that were demonstrated in various programming books - most of which were graphics programming related. This was fine at the time, and I certainly learned a lot over the years, but I never really took too much time to dive into C++ or some of the nuances of its deterministic memory model.
As I mentioned a couple posts ago, I have really been digging in deep into C++11/14, and trying to gain a deeper insight into why and how certain paradigms are considered good practice, while others are horribly bad practice. The primary problem with all of the pointers that I described above is that I was unwittingly defaulting to using reference semantics everywhere... As far as correctness goes, that isn't really such a big deal - you can of course write correct programs that use only reference semantics (C# and Java use reference semantics almost exclusively). However, C++ gives you a bit more freedom to choose how and when your objects are treated as references or values, so it is worthwhile to really consider how your classes will be used before you write them.
The obvious choice in reference vs. value semantics is determined when you declare the variables for your objects. If you use pointers or (to a lesser extent) references, then you are clearly choosing reference semantics. However, your actual class design itself also plays a big role in defining the semantics of how it gets used. All of the copy control methods (copy constructor, copy assignment operator, destructor, and the move constructor/move assignment operator) essentially prescribe how your class instances are moved around in a program. And as an engine author, you have to choose ahead of time how that should look. I think this single design choice has one of the biggest impacts on how you (and your users) work with your software.
Once again returning to the pointer default... This default is what most beginning C++ developers choose, and it works fine. However, once you start expanding to multithreaded programming, always using pointers can begin to complicate things. If you have the chance to use value semantics, making a copy of a value for a second thread becomes trivially easy - but if you are using reference semantics it is a bit more tricky. You can make a copy of your references, but all the references still point to the same instance - so multithreaded programming gets even more complex than it already is.
In addition, on modern processors, memory access is THE main bottleneck when trying to fully utilize a processing core. If you are using reference semantics, you are inherently less cache coherent than if you use value semantics (I know, I know, there are some cases where it is better to use reference semantics, but in general this is the exception and not the rule). Creating an array of value objects is simple - you just make it, and the objects are initialized how you indicate during their instantiation. They are destroyed when they go out of scope. On the other hand, creating an array of reference objects is more complex... Are they valid references? Can I dereference them now? When do they get initialized? When can they be destroyed? Did I already destroy them? Lifetime management is just plain easy with value semantics, and not as easy with references.
So to get started with HG4, I am taking some extra time to consider a general set of guidelines for object management with this new understanding in mind. Sometimes reference semantics are necessary (i.e. COM pointers force certain reference semantics, and Direct3D is full of COM...) so the real key is to figure out when you can use values, and when you should use references. It truly seems like a bit of an artistic process - the good C++ developers are very good at making these types of design choices. I still find it a bit taxing to come to a good solution, but when you get there, it sure does shine through and makes your API much easier to work with. Let's hope that I can get the initial design right, and then grow organically from there.
I came exactly to the same conclusion, not long ago.
I'm also rewriting my engine, switching from pointer and reference semantic, to what I call handle semantic.
Instead of a "pointer to", I now usually return an "handle to".
The handle (which usually is a simple 32 or 64 bit unsigned int) is easily copyable, movable and storable in linear array, and helps simplifying multithreading programming.