As I started using Unity I became more curious about the things I didnt know. I realize how little my knowledge is in terms of game development. I also realize that using an engine is the way to go. But the thing is, I started to have this hunger to know how do this people do this engine. I have this game in my mind and I am currently doing it. But I am getting side track by this thoughts about how things work. It seems that my interest keeps on changing but my goal is still the same. To make the game I had in mind. Right now I am thinking if I should buy the book about game engine. A moment ago I was searching about scene graph and other stuff which I shouldn't worry about since I am using Unity. Has anyone been like this before?

There is much to know, and much to learn. There is much material online you can learn from, but it is self-directed and you need to sift through what is most important at every phase of your learning. If you're looking for a dead-tree book on Unity there are several including the one linked to at the bottom of my post. Reviewed books tend to be better if you need guidance of what to learn next; if you don't know what you need to know next it can help get you to the level of basic competency.

As for what goes inside game engines, they are complex beasts. Consider that the major engines have been in development for years and tend to have large teams behind them. They typically represent several hundred work-years of effort by professionals, each person being expert in their own domains. As such, you could not create an engine like Unity or Unreal by yourself if you spent one hundred or five hundred years on it. They contain a huge number of systems: graphics processing, graphics file processing, graphics display, audio playback, audio formats, physics-based an non-physics based simulations, math and numerics libraries, disk libraries, animation libraries, UI libraries, networking support, and much more.

Has anyone been like this before?

I often find new things I know nothing about, and I solve that by reading about it at the Internet, or from a book. Pondering how I would build it, also tends to work.

I also find things I want to try. Usually I manage to delay that until a good excuse comes along to spend time on it :)

The world is full with wonderful, weird, good, and bad stuff. No way anyone is going to know it all, but it sure is fun exploring random subjects. I also had the experience that I invented a solution, which I found later, already existed. It's always nice to see that other people reach the same conclusion as you did, albeit lots of years earlier :)

PURSUE YOUR INTERESTS!

My interest is conflicting with my goal really. That's my I am more confuse

You should probably figure out what you really want to do. I tried Unity for about 6 months. It's awesome, but I discovered it's not really for me, not as my primary focus anyway. I found myself buying not only art assets but a lot of code as well. With my previous experience I was able to debug the code I was buying and make it work correctly. I was certainly understanding it. But I quickly came to miss the low level understanding of things. And I was buying more than I was learning. I missed the learning.

For me personally, I need to know things from beginning to end and build my own engine. Part of what I truly love about this is discovering the deep mysteries of this stuff. I went from DX11 to Unity and I found that I missed doing things from scratch and the discovery of the low level processes. My goal is not to finish a game, not in the near future, and not until I'm ready. Instead it is to understand how it all works and develop my ability with the craft.

So, I'm doing C++ and OpenGL now. I wouldn't hesitate to write a prototype with Unity or test something in Unity. But I love coding it from scratch and learning how to code it from scratch. This is not the right path for everyone; in fact, it's probably not the right path for most people. But it's the right path for me.

Maybe give OpenGL a try. Maybe you'll find it's not for you. Maybe you'll go back to Unity. Maybe you'll do something else instead. But at least you'll know more about what is right for you.

http://learnopengl.com/

You should probably figure out what you really want to do

I tried some frameworks in order:

XNA -> SFML -> Opengl (I used the website you linked) ->Monogame -> Unity

I guess I could summarize my confusion like this:

When I was using XNA, it was always exciting as I always learn something new especially browsing some source code. It was great and fun. Especially when I started opengl, since this is where I realize how things work using the website you gave, but even this is just scratching the surface. But then my mind started to change again, like many aspiring game devs they all wanna work at some big studios, so am I. So there is this doubt that even I am able to learn the low level stuff, it is the result that people are looking for. and basically working on an engine alone is such a very time consuming task. Then I started using Unity. 1st month I was having fun. ITs easy. All you gotta do is think about ideas and then plan the things you gotta do. But then I started to missed the low level stuff. the unknown things you discover and say "aaaahhh so this is how it works". So I guess I am on the same boat as you. I maybe missing the excitement. The kind of self achievement thing. Or maybe I am just bored. I don't know.