Jump to content
  • Advertisement
Shaarigan

Engine Core Features

Recommended Posts

So here again with some more opinion/discussion related topic about what features should/are normally be related into a game engine's core system from the implementation side of view. With a huge time I invested into writing and even more into research for my very own game engine project, there were as many differences as there are to coding guidelines. Especially open source engines (Urho3D, Lumberyard) and those that offer there source code (Unreal, CryEngine, Phyre) change during the centuries depending on current technical, coding and hardware standards. They all consist of third party code and libraries that are set apart from the core system bur rely on it.

Now I'm on a point where I want restructure/cleanup my project(s), removing obsolete and integrate prototyping code as same as change the growing code base to have a better clean managed file structure. The question that has had to emerge for long or short is what parts of the code base should be a core feature in order to keep the core system clean and flexible. Again reading many cross references it pointed some similarity out but also huge differences (like Unreals hashing/encryption code beeing in core) so I made a list of important stuff any game needs (I left out platform specifics and audio/renderer here because they are platform dependent and should stay in an own module/system)

  • Allocator (memory management)
  • Math (Vector, Matrix, Quaternion ...)
  • Threading (Threading, Locks)
  • Container (apart from the STL, Queue, Buffers, certain types of Array ...)
  • String (management, encoding, utils)
  • Input (seen a game without input?)
  • IO (reading/writing files and manage filesystem structure)
  • Type Handling

And a list that is optional but may/may not be inside the core code

  • Logging (because it is a development tool)
  • Profiler (see logging)
  • Serialization
  • Plugins
  • Events

In my opinion, logging and profiler code is a heavyly used feature in development but is stripped out mostly when deploying a release version, not all games rely on events rather than using other interaction (like polling, fixed function loops) and also the same for plugins and serializing/deserializing data; so what do you think should also be a must have in core system or should go from the list into core system as well as a must have? What else should go outside as an own module/system (into an own subfolder inside the engine project's code base)?

Courious to read your opinions ;)

Edited by Shaarigan

Share this post


Link to post
Share on other sites
Advertisement

Cross platform GPU wrapper (preferably low level in the core module - with models/scenes/materials as a higher level module), Audio, 3D collision and rigid body dynamics are all just as core as input (for most games). Though yes these can be modules that live alongside the core... But in that case, input should be too.

A networking module needs friendly sockets, HTTP, SSL and encryption (some platforms require all traffic to be encrypted).

Algorithms for hashing, etc, make sense in the core. e.g. The hash map will reference them!

I don't have a string class in my engine; it's a good way to discourage people from using strings :P 

Under threading, I'd have a thread-pool and job graph, not individual threads and locks.

Core game IO should be a lot more limited than generic IO. Loading of named assets, and reading/writing profile / savegame files (no general OS disk access). These are really two different things - not two uses of one IO library. e.g. assets are always streamed asynchronously from known formats and never written to. Savegames are always stored in some OS-dependent path such as "my documents"/appdata/home/etc.

Profiling, logging, assertions are mandatory development tools required to do your job, so should be in the core. Same goes for a good memory manager with debugging features. I'm leaning more and more towards having something like ImGui and a line rendering API as core, simply for development/debugging purposes. 

If you plan on using a Scripting language (or even reloadable C++), a binding/reflection system makes sense as it will be pervasive. You can also use these bindings to generate serialisation/visualisation functions.

Share this post


Link to post
Share on other sites

Forgot the memory management/allocator stuff so added it to the list :P I'm on the hop to force strict memory management at all levels of the engine to at least throw ya an assertion failure arround the head when leaving anything on the heap on scope exit.
 

59 minutes ago, Hodgman said:

Cross platform GPU wrapper (preferably low level in the core module - with models/scenes/materials as a higher level module)

I think you mean the graphics API (DX, GL, VK, whatever) here?

1 hour ago, Hodgman said:

3D collision and rigid body dynamics

Highly depends on the game, Puzzle or 2D platformer might not need any of these so I see them in an optional external instead of core where I'm not sure about input system here (what is a reason to write this post). But I think you were right and it depends a level upward than placing it in core directly.

Networking should stay external as not every game needs it but as I have seen for example Unreal hosts its HTTP stuff inside the core module. I personally would avoid this and instead keep networking low level on the one hand (to have possibility to write also server code) but also more high level as a game client.

Same thoughts for the crypto module. It consist of a lot of static functions to utilize AES, ECDSA, XTEA as same as different hashing functions SHA, RIPEMD and MD5 (as short hash) where the core system contains some text related 32/64 bit hashing functions used for example to identify names.

I agree on your arguments about strings and IO in general. Assets should be converted to packages of engine preprocessed data and savegames should be serialized/ deserialized in order to keep a level of error proofness but these systems may need to interact themselfs with the disk/ network/ cloud storage so I keeped this in my entire list ;)Nearly anything inside my code is already using streams/ buffered reading and static shared memory (aka memory mapped files) as needed and passing a stream interface is preferreable by any data processing function.

Under threading I have currently threads and locks as utility code wrapping the underlaying OS APIs where my JobPool and event system is in an own module but I see that this might go as well into core too. What do you think about event driven engines vs. polling/ update driven ones?

Profiler and logging were currently own modules as I thought of having them in build while development but strip them out when shipping. Profiler may be a point of diskussion but I see logging at least for support purposes as a core feature now so maybe move it there. Dont know about profiler yet :)

Reflection is currently realized as an external module named MetaType that is intendet to provide some meta information and C# like function invocation (for runtime class manipulation) as same as some serialization frontends. As I have seen this in core for various projects, I will potentially move it into core level, for at least support scripting and maybe editor UI interaction from C# level.

Share this post


Link to post
Share on other sites

Hardware/OS specific items should still have an interface layer in your core such that it is consistent no matter how you implement the backend.  In regards to that, you really should add input device handling (keyboard, mouse & joystick) and window management items.  Depending on goals, a lot of folks roll window management into the rendering engine but I tend to think it should be separated for quite a few reasons.  Just my $0.02.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Advertisement
  • Advertisement
  • Popular Tags

  • Popular Now

  • Advertisement
  • Similar Content

    • By BearishSun
      bs::framework is a newly released, free and open-source C++ game development framework. It aims to provide a modern C++14 API & codebase, focus on high-end technologies comparable to commercial engine offerings and a highly optimized core capable of running demanding projects. Additionally it aims to offer a clean, simple architecture with lightweight implementations that allow the framework to be easily enhanced with new features and therefore be ready for future growth.
      Some of the currently available features include a physically based renderer based on Vulkan, DirectX and OpenGL, unified shading language, systems for animation, audio, GUI, physics, scripting, heavily multi-threaded core, full API documentation + user manuals, support for Windows, Linux and macOS and more.
      The next few updates are focusing on adding support for scripting languages like C#, Python and Lua, further enhancing the rendering fidelity and adding sub-systems for particle and terrain rendering.
      A complete editor based on the framework is also in development, currently available in pre-alpha stage.
      You can find out more information on www.bsframework.io.

      View full story
    • By BearishSun
      bs::framework is a newly released, free and open-source C++ game development framework. It aims to provide a modern C++14 API & codebase, focus on high-end technologies comparable to commercial engine offerings and a highly optimized core capable of running demanding projects. Additionally it aims to offer a clean, simple architecture with lightweight implementations that allow the framework to be easily enhanced with new features and therefore be ready for future growth.
      Some of the currently available features include a physically based renderer based on Vulkan, DirectX and OpenGL, unified shading language, systems for animation, audio, GUI, physics, scripting, heavily multi-threaded core, full API documentation + user manuals, support for Windows, Linux and macOS and more.
      The next few updates are focusing on adding support for scripting languages like C#, Python and Lua, further enhancing the rendering fidelity and adding sub-systems for particle and terrain rendering.
      A complete editor based on the framework is also in development, currently available in pre-alpha stage.
      You can find out more information on www.bsframework.io.
    • By Gnollrunner
      Hi again,  After some looking around I have decided to base my game directly on Direct X rather than using an existing game engine.  Because of the nature of the stuff I'm doing it just didn't seem to fit very well and I kept running into road blocks.  At this point I have a big blob of code for doing fractal world generation and some collision code,  and I'm trying to put it into some form that resembles a game engine.  Since I've never used one before It's a bit alien to me ..... so can someone direct me to a book, website, article, whatever... that covers this?  I'm mainly looking for stuff that covers C++ library design. I'm not adverse to using 3rd party tools for stuff I can used them for.
    • By chiffre
      Introduction:
      In general my questions pertain to the differences between floating- and fixed-point data. Additionally I would like to understand when it can be advantageous to prefer fixed-point representation over floating-point representation in the context of vertex data and how the hardware deals with the different data-types. I believe I should be able to reduce the amount of data (bytes) necessary per vertex by choosing the most opportune representations for my vertex attributes. Thanks ahead of time if you, the reader, are considering the effort of reading this and helping me.
      I found an old topic that shows this is possible in principal, but I am not sure I understand what the pitfalls are when using fixed-point representation and whether there are any hardware-based performance advantages/disadvantages.
      (TLDR at bottom)
      The Actual Post:
      To my understanding HLSL/D3D11 offers not just the traditional floating point model in half-,single-, and double-precision, but also the fixed-point model in form of signed/unsigned normalized integers in 8-,10-,16-,24-, and 32-bit variants. Both models offer a finite sequence of "grid-points". The obvious difference between the two models is that the fixed-point model offers a constant spacing between values in the normalized range of [0,1] or [-1,1], while the floating point model allows for smaller "deltas" as you get closer to 0, and larger "deltas" the further you are away from 0.
      To add some context, let me define a struct as an example:
      struct VertexData { float[3] position; //3x32-bits float[2] texCoord; //2x32-bits float[3] normals; //3x32-bits } //Total of 32 bytes Every vertex gets a position, a coordinate on my texture, and a normal to do some light calculations. In this case we have 8x32=256bits per vertex. Since the texture coordinates lie in the interval [0,1] and the normal vector components are in the interval [-1,1] it would seem useful to use normalized representation as suggested in the topic linked at the top of the post. The texture coordinates might as well be represented in a fixed-point model, because it seems most useful to be able to sample the texture in a uniform manner, as the pixels don't get any "denser" as we get closer to 0. In other words the "delta" does not need to become any smaller as the texture coordinates approach (0,0). A similar argument can be made for the normal-vector, as a normal vector should be normalized anyway, and we want as many points as possible on the sphere around (0,0,0) with a radius of 1, and we don't care about precision around the origin. Even if we have large textures such as 4k by 4k (or the maximum allowed by D3D11, 16k by 16k) we only need as many grid-points on one axis, as there are pixels on one axis. An unsigned normalized 14 bit integer would be ideal, but because it is both unsupported and impractical, we will stick to an unsigned normalized 16 bit integer. The same type should take care of the normal vector coordinates, and might even be a bit overkill.
      struct VertexData { float[3] position; //3x32-bits uint16_t[2] texCoord; //2x16bits uint16_t[3] normals; //3x16bits } //Total of 22 bytes Seems like a good start, and we might even be able to take it further, but before we pursue that path, here is my first question: can the GPU even work with the data in this format, or is all I have accomplished minimizing CPU-side RAM usage? Does the GPU have to convert the texture coordinates back to a floating-point model when I hand them over to the sampler in my pixel shader? I have looked up the data types for HLSL and I am not sure I even comprehend how to declare the vertex input type in HLSL. Would the following work?
      struct VertexInputType { float3 pos; //this one is obvious unorm half2 tex; //half corresponds to a 16-bit float, so I assume this is wrong, but this the only 16-bit type I found on the linked MSDN site snorm half3 normal; //same as above } I assume this is possible somehow, as I have found input element formats such as: DXGI_FORMAT_R16G16B16A16_SNORM and DXGI_FORMAT_R16G16B16A16_UNORM (also available with a different number of components, as well as different component lengths). I might have to avoid 3-component vectors because there is no 3-component 16-bit input element format, but that is the least of my worries. The next question would be: what happens with my normals if I try to do lighting calculations with them in such a normalized-fixed-point format? Is there no issue as long as I take care not to mix floating- and fixed-point data? Or would that work as well? In general this gives rise to the question: how does the GPU handle fixed-point arithmetic? Is it the same as integer-arithmetic, and/or is it faster/slower than floating-point arithmetic?
      Assuming that we still have a valid and useful VertexData format, how far could I take this while remaining on the sensible side of what could be called optimization? Theoretically I could use the an input element format such as DXGI_FORMAT_R10G10B10A2_UNORM to pack my normal coordinates into a 10-bit fixed-point format, and my verticies (in object space) might even be representable in a 16-bit unsigned normalized fixed-point format. That way I could end up with something like the following struct:
      struct VertexData { uint16_t[3] pos; //3x16bits uint16_t[2] texCoord; //2x16bits uint32_t packedNormals; //10+10+10+2bits } //Total of 14 bytes Could I use a vertex structure like this without too much performance-loss on the GPU-side? If the GPU has to execute some sort of unpacking algorithm in the background I might as well let it be. In the end I have a functioning deferred renderer, but I would like to reduce the memory footprint of the huge amount of vertecies involved in rendering my landscape. 
      TLDR: I have a lot of vertices that I need to render and I want to reduce the RAM-usage without introducing crazy compression/decompression algorithms to the CPU or GPU. I am hoping to find a solution by involving fixed-point data-types, but I am not exactly sure how how that would work.
    • By babaliaris
      Well i found out Here what's the problem and how to solve it (Something about world coordinates and object coordinates) but i can't understand how ti works. Can you show me some examples in code on how you implement this???
       
      Scaling Matrix:
      m_Impl->scale = glm::mat4(1.0f); m_Impl->scale = glm::scale(m_Impl->scale, glm::vec3(width, height, 0)); Verticies:
      //Verticies. float verticies[] = { //Positions. //Texture Coordinates. 1.0f, 1.0f, 0.0f, 0.0f, 2.0f, 1.0f, 1.0f, 0.0f, 2.0f, 2.0f, 1.0f, 1.0f, 1.0f, 2.0f, 0.0f, 1.0f }; Rendering:
      //Projection Matrix. glm::mat4 proj = glm::ortho(0.0f, (float)window->GetWidth(), 0.0f, (float)window->GetHeight(), -1.0f, 1.0f); //Set the uniform. material->program->setUniformMat4f("u_MVP", proj * model); //model is the scale matrix from the previous code. //Draw. glDrawElements(GL_TRIANGLES, material->ibo->GetCount(), GL_UNSIGNED_INT, NULL);  
      Shader:
      #shader vertex #version 330 core layout(location = 0) in vec4 aPos; layout(location = 1) in vec2 aTexCoord; out vec2 texCoord; uniform mat4 u_MVP; void main() { gl_Position = u_MVP*aPos; texCoord = aTexCoord; } #shader fragment #version 330 core out vec4 colors; in vec2 texCoord; uniform sampler2D u_Texture; void main() { colors = texture(u_Texture, texCoord); }  
      Before Scaling (It's down there on the bottom left corner as a dot).

       
      After Scaling

       
      Problem: Why does the position also changes?? If you see my Verticies, the first position starts at 1.0f, 1.0f , so when i'm scaling it should stay at that position
  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

Participate in the game development conversation and more when you create an account on GameDev.net!

Sign me up!