Make an engine from scratch's resources?

Arguably the basics you need for a game engine would be a language, a compiler, and then the "engine" itself(code of course,) which I would say at the very least needs some way to interact with the user. From there we can say that the simplest game engine would be you creating an interactive simulation that can somehow show things to the user and react to their input.

Or put more simply, decide what you want your engine to do and get the basic steps to get it going. Want to make a 3d game engine centered around some kind of FPS? Well you'll want to get a game loop going and set up rendering and input handling. From there its all cherries on top really. Figure out how you want maps laid out, create or pick some tools to create world geometry, load it in, display it, add collision detection.

Engine is just a descriptive word for a bunch of common code, you can't make an engine without having some kind of game type in mind at least.

If you don't have anything in mind then maybe you should try making a simple game instead, make pong or something. Making a game shows you a lot about what an engine needs to be able to do and you'll have to research what you need to do it. But at the very least you need a compiler, and an IDE is very useful.

Ya im planning on something like a fps, but I want to build it from scratch and not use Unread engine or Cry-engine or stuff for this.

You just need a compiler/IDE for your language of choice, e.g. Microsoft Visual Studio for C++.

You'll then use that to create all the other tools that you require...

I would like to know what programs I would need to make my own engine from scratch. Yes I know it is highly un-recommended to make one from scratch if your just by yourself... But I've been learning multiple variations for multiple scripting languages. I am motivated to put in all the effort and I have alot of free time to do this. ( I know it will probably take me years ). I want to be able to control all the vertices of the models and know exactly all my limits of my engine. I don't want any criticism or asking why would you do this... I just want a slap into the right direction

I decided to do something similar for the purposes of learning, and I must say, you really do learn far more with this route than the typical "use a pre-existing game engine" route. For your own sanity though, I would strongly suggest using some 3rd party libraries until you have at least some game engine programming experience under your belt. Some libraries that you may want to look in to are: SDL2/SFML/GLFW (Windowing/Input), OpenGL/DirectX (Rendering), irrKlang (Audio), Assimp (Model Loading), Bullet/PhyX/Havok (Physics), GLM (Math), and GLEW if you're using OpenGL. Of course, use Visual Studio 2013/2015 with C++. Look into Trello and Github as well if you haven't already done so.

I personally believe it's better to tailor an engine for a specific game that you want to make so you have some realistic endpoint (and you have some semblance of focus, which is extremely important in project management). Once you've completed the game, you can throw away as many designs as you want, and keep the designs you liked when you start a new project. I guess what I'm saying is that the best way to evaluate your engine is to make a game with it, and following the process of game engine -> game -> next game engine iteration -> next game is far more effective than simply working on an engine. I think this is the process that people are referring to when they say to "make games, not engines" but hey, I could be wrong.

Good luck!

You just need a text editor or IDE, a compiler, a language, a brain, and patience. Oh and a large library of technical books helps too.


As for engine design. I don't recommend starting off programming an engine if it's your first time. Just do some cmd prompt games first.

If you are going to go ahead and design the engine, I personally recommend designing the engine for the genre you want. It's easier to adapt the engine that way, and there is less code to throw out. The problem is... that it's going to take longer. But it's a learning experience. As I am learning that myself.

Although I think it might be hard for absolute beginners (but I have no idea about your level of C++ expertise) I'd say www.rastertek.com does just what you want.
I also went through the entire tutorial and am in the never ending process of writing an engine although I'm now slowly thinking game rather than engine and will add more to it accordingly.
After rastertek I have learned a few useful libraries like Bullet, Assimp etc as suggested above.
Trying to use them in rastertek tutorials is an excellent exercise btw.

As an aside, I started with rastertek knowing very little about classes. My knowledge of C++ was purely procedural never truly knowing when or for what purpose to put something in a class and it's doing these tuts that not only taught me DirectX but also basic OOP. In the meantime I kept reading about C++ and DirectX from as many sources as possible.

About 10 years ago I started making my own engine from scratch. The way that I did this was based on what I was planning on doing with the engine at the time. I knew that I was going to be doing medical simulators for work, so I started building my engine around that. When that project was done, and I needed to start working on the next, I had my engine code base as a starting point for the next one. After 10 years of projects I now have an engine that has a bunch of capabilities that came from using it for different tasks.

So my advice to you would be, start small, and start with a project in mind. Build that project to completion without worrying too much about the engine. Then pick another project to work on, but this time start with the engine you already have and improve upon it.

I made the decision you did the over ten years ago now using C++, OpenGL, DirectX and some other stuff. Although my then code structure is embarrassing I actually got a good 3D game going that a few people actually played on Windows Millenium and XP. If you are curious,

https://github.com/ButchDean/AntiVirusGame2

Should give you an idea at least of the effort involved from a realistic standpoint to get it done.

As I am beginner to I did think about this. The problem I see to develop a engine as a novice is. Your produce something where the end users are not gamers but fellow gamedevelopers.
So as I am alone to, these users are the artist scripters. Not fellow coders.
Which means already starting with a script engine and a content pipeline is a huge task.
These are heavy features.
Also it make sense to have a game that shows of most game engine features.
And as of being alone, for your show of example game, you need to take these roles to.

But instead of a full game I would go for full featured gameplay demo.
I have seen the C4 game engine architecture. And it seams there goes a lot of modules and systems into a full fledge engine.

So the advice I read make sense to just make games and start simple. A reusable framework will emerge after some refactoring and with each follow up game it grows in features until it could be called a small engine.

As I am alone and tent to be for long time as a lone hobby programmer. I also avoid the heavy need for artist or a team if I would scale up so I prefere to stick with more procedural content generation.

Thinks I would avoid to implement in my first simple game would be.
Content pipeline
Script engine
Resource cache
Resource manager
Advance memory management.
No mplay

So stick first with the very basic parts for very basic game.

Some libraries that you may want to look in to are: SDL2/SFML/GLFW (Windowing/Input), OpenGL/DirectX (Rendering), irrKlang (Audio), Assimp (Model Loading), Bullet/PhyX/Havok (Physics), GLM (Math), and GLEW if you're using OpenGL.

This is the opposite of “from scratch”.
This is exactly how not to learn anything while making your own engine.

Whenever I see people bragging about having made their own engines, and then they list off all the libraries they used—“I used this for graphics, that for sound, those for math, this for physics, etc.”—I just have to face-palm and ask, “So what part of the engine did you actually write? What part of this engine is actually yours?”.
(Of course you need either Direct3D or OpenGL; you should have replaced those with Ogre or Irrlicht Engine.)


If you’re going to write your own engine, write your own engine.
Yes it takes time and yes it will be crap your first run through, but the point is to explore all these low-level technologies on your own and learn more than the average bear about how games work.


All that being said, there may eventually be a time when you want to use some libraries. But that time comes only after you’ve already made your own implementation(s). For example, now I use FreeImage, but I used my own loaders for .BMP, .GIF, PNG, and .TGA for many years first (I still use my own for .DDS and .LSI).
Now I might start using Bullet, but I already wrote a decent physics engine years ago.



So the real answer is:

You just need a compiler/IDE for your language of choice, e.g. Microsoft Visual Studio for C++.

You'll then use that to create all the other tools that you require...

L. Spiro

All that being said, there may eventually be a time when you want to use some libraries. But that time comes only after you’ve already made your own implementation(s). For example, now I use FreeImage, but I used my own loaders for .BMP, .GIF, PNG, and .TGA for many years first (I still use my own for .DDS and .LSI).

Whereas I agree with most of what you said above, I think this is perhaps pushing it a bit. I've seen countless senior programmers on this board telling others not to reinvent the square wheel. I mean where does it stop? Should you have LS_wsprintf(), LS_itoa(), LS_strcat()?
It's of course a very good exercise for purely academic reasons but for most such low level functionality I do use libraries or API functions.
And this is because I cured my nimby syndrome after reading lots of advice against the practise.
I did write an absolutely primitive bmp texture loader just to be sure that I understood what is passed to mappedSubResource.pMem but I will not use it in a game. For this I use DDSTextureLoader.

Edit: I meant NIH syndrome, not NIMBY obv.

Disclaimer: My job and interests do not necessarily apply to everyone. There is a gradient. What I say here may feel entirely compatible with your personal feelings, partially compatible, or not-at-all compatible. Take it as just that. I will explain why I did things and how they impact me today, but since my main interest is in making low-level technology for use in many games rather than game logic for use in just a single game, take it influentially based on where you fall in that same spectrum (if you want to do the same low-level stuff I do then listen up, if you are mildly interested, listen up but don’t assume you have to follow the same path, and if you are completely uninterested, just ignore it.)

Should you have LS_wsprintf(), LS_itoa(), LS_strcat()?
It's of course a very good exercise for purely academic reasons but for most such low level functionality I do use libraries or API functions.

I have rewritten those functions, yes. Every once-in-a-while a person needs to be born who wants to spend time re-investigating even all these simple low-level things, but even if not just out of interest there are benefits to doing it.
Thanks to my interest in the low-level workings of things I tried to rewrite these things back when I was 14, and needless to say my results were sub-par. That didn’t deter me because while making my own functions faster was a personal challenge, the actual goal was just to practice my logic and coding skills, and if my results were inferior in performance, I would also have the motivation to understand the underlying architecture better to know why.

Maybe around 16 I made a second version of many of the C-library functions and this time a few of my functions had the same performance as the originals. I could tell that I was growing and understanding the underlying architecture better. Since I was able to gain experience in these areas not just by rewriting these functions over and over but by working on other projects such as games or whatever interested me, I grew in regular game development at the same time.

Naturally I started my actual career as a regular game programmer. No problem. I learned just as much about standard game development as I did about low-level workings over the same period of time.
Slowly I showed people that I was interested in those low-level things, and in the middle of my career I was working as both an engine developer and also a game developer, building the game on top of the same engine I was developing.

Since leaving that company, I have shifted entirely into engine development, previously working on tri-Ace’s ASKA engine, and now working at Square Enix as a senior graphics programmer on their Luminous Engine.


You give examples of low-level C-library functions as examples of the mundane—a cut-off point where you should question whether it is sane or not to continue.
It’s because of my experience with these areas that I have the career that I have.

This might be a straight-forward function to you:

CMatrix4x4 & MatrixRotationX( float _fAngle ) {
	float fS = sinf( _fAngle );
	float fC = cosf( _fAngle );
	_11 = 1.0f; _12 = 0.0f; _13 = 0.0f; _14 = 0.0f; 
	
	_21 = 0.0f; _22 = fC; _23 = fS; _24 = 0.0f;
	
	_31 = 0.0f; _32 = -fS; _33 = fC; _34 = 0.0f;
	
	_41 = 0.0f; _42 = 0.0f; _43 = 0.0f; _44 = 1.0f;
	return (*this);
}

Because I took the time to learn about low-level things while also learning general programming and game-programming long ago, to me several things immediately glare at me.

#1: Never use sinf() and cosf() on the same value. Use sincosf(). Not implemented? Well:

	LSE_INLINE LSVOID LSE_FCALL CMathLib::SinCos( LSREAL _fA, LSREAL &_fSine, LSREAL &_fCosine ) {
		__asm {
			fld		_fA
			fsincos
			mov		ecx, _fCosine
			mov		edx, _fSine
#if LSM_BASETYPE == LSM_FLOAT
			fstp	dword ptr [ecx]
			fstp	dword ptr [edx]
#elif LSM_BASETYPE == LSM_DOUBLE
			fstp	qword ptr [ecx]
			fstp	qword ptr [edx]
#endif
		}
	}

Works with doubles and floats. Several times faster, and equally accurate, as calling sinf() and cosf() on the same value.

But wait, I’m still not satisfied.
If I can make a replacement for sinf() and cosf(), each being several times faster than the originals, I could also make a custom sincosf() faster than the original.
Using a Taylor series as suggested by Robin Green I get these results:

Testing CMathLib::Cos().
7371162 27971263
Difference: -20600101
0.26352623105049133 (3.7946886578 times faster)
CMathLib::Cos() beats ::cosf().
 
Testing CMathLib::Sin().
7766147 28035014
Difference: -20268867
0.27701598405838013 (3.6098999969 times faster)
CMathLib::Sin() beats ::sinf().

My custom sincosf() is about 4 times faster than fsincos.


Today at the office I found that a lot of our matrix routines are using the original code I posted above, using sinf() and cosf() separately on the same value. This is something I am going to fix on Monday. Yes, it really happens inside even the biggest of studios because so many people just say, “Why re-invent the wheel?”.


You can’t use those low-level C-library functions as an example of what you would call the cut-off between sane and insane, because it just so happens that rewriting those exact routines was a part of my growth into the position where I am now. And I didn’t stop there, I questioned everything. memcmp, memcpy, sinf, cosf, strlen, you-name-it. If I fail to make a custom routine faster, I learn something, and if I make a custom routine faster, I learn something. And I have a new routine to use.
And none of the time I spent challenging myself with these kinds of rewrites came at the cost of anything else. It’s insane to say that I only know how to optimize things just because after each project I’d finished I’d challenge myself with a low-level C-library rewrite.


This is now a different tangent (now a personal rant not directed at any specific person currently involved), but I do get tired of seeing people advising others to, “make games, not engines.”
It’s like telling a gay guy to date women, not men.
Look-
If you’re interested in making engines, you probably have your reasons, and while people tend not to ask what those are, they also assume they are the wrong reasons based on their own personal preferences (which tends towards making games).

Not everyone is like you, and when you saturate the “Advice Channel” with your own views you really aren’t helping people “come out of the closet”.
I, like many people, am naturally drawn towards the low-level, “How does this work,” spectrum. It’s natural. It can’t be helped.
So telling me to, “make games, not engines,” is just sending confusing signals, just as it would be to tell a gay guy to date women, not men.

Either way, it’s not a choice. I can’t control my desire to make engines any more than a gay guy can control his urges for men.
So (meat of the personal rant here) enough with the frigging “do things my way” advice already.

Frigging figure out what you want and just do it. At least the topic poster knows for-sure that he or she wants to make an engine. Step #1 is admitting you don’t have a problem.


L. Spiro