So, a few question, is, how do you develop game engine? (cue in the laughtrack)
Not easily, You should start by making a game and then refracturing the reusable parts.
I would suggest using libraries as much as possible to save you time and effort, Examples: Bullet Physics, Hieroglyph3 (Graphics), Ogre(Graphics), QT(UI), WxWidgets(UI), RakNet(Networking), Etc.
Bottom line: You will be able to build a (complete) game faster/easier using Unity/UE then building it from scratch.
I was building my own engine, Basic but usable.
Then I got UE4.10 and was able to recreate my game within a week (Check out "Toy on a Mission" Week of Awesome II).
Since then I have been adding stuff and features that would not be possible in such a time frame because I would have to figure it out and implement it.
Sure Unity/UE have there problems and quirks, But you need to decide what would be better for you.