Are you testing in release or debug?
You need to profile the two versions to know what's going on. How much CPU time is being spent and in what functions, before and after?
Just to answer the questions I mostly do everything in debug mode. Turns out running the profiler when I"m using the shared pointers about 32% of my program execution time is spent in std::swap<GUIElement *> whereas with the raw pointers no single call gets above 5% of execution time (_RTC_CheckStackVars is the highest if you care to know).
Anyway gives me somewhere to consider. Although having read everyones input here I think what I need to do is look more closely at how I'm passing around my GUIElements and verify how/when my allocations and deallocations *should* be occuring.
Thank you all...
I see. Ok, so I have 2 pieces of advice for you:
1. Dont ever guess as to how some code will or is performing. Always profile, either with dedicated profiling tools or with your own timing code. Also, you need to profile your code with as close to final data as possible.
2. Dont ever do performance testing in debug mode. I made this bold so you will pay special attention to it. You should test performance in either release mode, or whatever build mode you have dedicated to shipping or perf testing your game. But a lot of times people just have debug and release, in which case you use release.
So before you seek other advice on this, or adjust your code in any way, stop what you're doing and test in release mode. Any framerate or timing data you have from debug mode is completely meaningless. I can say that without knowing anything about your code.