Sign in to follow this  
Plotnus

Game Engine Creation: What are the biggest challenges faced?

Recommended Posts

What are the biggest issues/challenges/difficulties faced when creating a game engine?

//copy of my post#3 for late comers so its not missed
The purpose of this question was to see what issues ore commonly faced so i can plan for them in my own design/implementation.

My goal to create an engine is just for a learning exercise, not commercial purposes.

Eg. I'd use what was stated here to decide what design patterns to use.
At this point I know that the Component and Flyweight patterns will be useful. (root scene node hierarchy, mesh data)

 

Edited by Plotnus

Share this post


Link to post
Share on other sites

What are the biggest issues/challenges/difficulties faced when creating a game engine?

 

Getting it to a point where you can actually create a game using it.

 

You're usually not going to have a lot to show for a game engine other than a handful of tech demos, especially during the first few months (years?!) of development, and that necessitates a lot of self-motivation. And unless you have some concrete goals in mind and a realistic scope for your engine, i.e. what kind of game(s) it's designed to be used on, features you want it to support, etc., development can likely linger on aimlessly because you never consider it "ready". That's why it's often recommended that one focuses on developing a few games first, and having the engine fall out of those based on commonly used functionality.

Share this post


Link to post
Share on other sites

Thanks for the input.

This is just for a learning exercise, not commercial purposes.

What issues did you face in the implementation?

The purpose of this question was to see what issues ore commonly faced so i can plan for them in my own design/implementation.

 

@zipster I see, so the danger of feature creep. I take your post as advice to have a realistically defined scope.

Edited by Plotnus

Share this post


Link to post
Share on other sites

One thing that a lot of "hobby engines" ignore is the toolchain / the real-world workflows of each individual user.

An engine needs to be user-friendly for all of it's clients, which includes:
* gameplay programmers
^ most hobby engines focus on this area
* AI programmers
* animation programmers
* graphics programmers
* network programmers
^ most hobby engines hard-code some solutions to these areas, instead of making flexible/extensible systems.
* level designers
^ most hobby engines have a terrible tool for this.
* 3D artists
* texture artists
* lighting artists
* UI artists
* effects artists
* animators
* musicians/composers
* sound engineers
^ most hobby engines completely ignore these clients!

The "hard part" is knowing how all of these people will be doing their jobs, and creating a workflow that keeps them happy and fits well with your engine.
Building the engine first often results in shitty workflows
-- Undetstand the toolchain first, then build and engine to match.

 

I personally wouldn't think of that being fully true. Most hobby engines are probably just about the same level as an indie project with an inhouse engine. You probably don't have a large enough team to build custom tools for all of that. So you typically build for what you know is the most likely.

 

Currently... I have a few plans.
A. TCP connect Blender to the game engine. Probably going to be a pain to do. Maybe not. If not, then life is good.
B. Separate tool chain. Reuse as much as possible. Level editor may use Sony's ASF if the documentation is a little better. Or may do it from scratch. Animations, and Models are custom fitted for the engine, and a plugin has been made for blender. Engine only uses Ogg and Thea for sound and video. Everything else? Formats already exist. Tools can be what ever you want.

 

Right now... I think the hardest part is trying to find a game framework system that works. My goal was to make it where entities can be split into tasks so they can be updated asynchronously with a scripting language. And allow it to be easily modified.

 

I thought about using an entity component system. But I quickly threw it out when I realized how limiting it actually was. And also setup is more tedious than people give it credit.

 

Adding new systems means you need to make some way to update it in the game. A number of systems will be created with only one or two things actually using it. It's not a one thing tackles all system.

 

I eventually decided to trade for GameObject-Component. With a specialty object for Level scripts. When the gameobject is created, it takes from a predefined archetype from file, creates it and registers what needs to be updated every pass. And what waits for event callbacks. It works great! Now came to finding a way to thread it!

 

I've thrown out code for this so often, and have a spiral filled with plans of attack. I origionally wanted as much of the engine in lua as possible. But then I realized that depending on the complexity of the game you are designing, it won't work well. Not to mention full Lua makes threading ridiculously difficult.

 

So... I threw out about 5,000 lines of code in one instance, another 1400 in another.

 

Now I'm looking into other options. Such as spawn a thread with a lua state attached to it, and treat Lua as script only. Swap to Angel Script. Etc.

 

I don't really have the resources, or the know how to do what big-daddy engines can do. I read through some of their code for ideas. But some of the sheer complexity makes it difficult to map out exactly what they are doing.

 

It's grown so frustrating, that I just might game logic single threaded.

 

Rendering. Resource Loading. Audio? All easy compare to the framework.

Edited by Tangletail

Share this post


Link to post
Share on other sites

You probably don't have a large enough team to build custom tools for all of that.

I didn't say you have to build custom tools for all that, but you do need to plan the workflows.
If you've got one artist and they want to produce models in Blender, textures in Photoshop and animations in Motion Builder, you need to build your engine around supporting these existing tools. If instead, an artist joins your team and you say "I've got an .obj importer!", they'll immediately turn around and walk back out the door laugh.png

 

e.g. The artist that I've paired up with for my current game is a die-hard XSI fanboi, so our engine supports using XSI as a modelling tool, material editor, rendering/lighting preview, scene editor and animator smile.png We also have things set up so you can export from XSI into the game while the game is still running for quick iteration.
 
Most hobby engines don't even have one artist on their team, so they have no idea what the content-creation workflow for their first game is going to be... So they create an engine, start making their first game, try to recruit people... and then... try to solve the content workflow issues that suddenly appear.

As well as answering "How do I write gameplay code?", you need to be able to answer "How do I create UI's?" and "How do I import a skinned mesh?", etc... Unfortunately this is often catch-22, as you want to tailor the answers to those questions to the talents and preferences of your content-creators, who you don't have yet.

So if your first game with the engine is going to be a one-man game, then first you've got to learn how to design levels and create art, and then create the engine around your own workflow preferences for those tasks.

Edited by Hodgman

Share this post


Link to post
Share on other sites

The biggest challenge is writing the tools and level editor to make the engine actually useful.

 

Writing an engine itself isn't actually difficult.  Its been done so much there are books and tutorials out there that it becomes just a case of following the recipe and adding your own space.

Share this post


Link to post
Share on other sites
The biggest challenge is writing the tools and level editor to make the engine actually useful.

It's not a challenge, it just takes monster time.

 

Writing an engine itself isn't actually difficult.

It's false, writing a good engine is not easy at all, you have to deal with threading, motion system, cross platform, easy to use, extensible.....

 

Its been done so much there are books and tutorials out there that it becomes just a case of following the recipe and adding your own space.

Lot of features, especially in motion system doesn't have any tutorial or books, it's pure research and takes time.

Edited by Alundra

Share this post


Link to post
Share on other sites

 

You probably don't have a large enough team to build custom tools for all of that.

I didn't say you have to build custom tools for all that, but you do need to plan the workflows.
If you've got one artist and they want to produce models in Blender, textures in Photoshop and animations in Motion Builder, you need to build your engine around supporting these existing tools. If instead, an artist joins your team and you say "I've got an .obj importer!", they'll immediately turn around and walk back out the door laugh.png

 

e.g. The artist that I've paired up with for my current game is a die-hard XSI fanboi, so our engine supports using XSI as a modelling tool, material editor, rendering/lighting preview, scene editor and animator smile.png We also have things set up so you can export from XSI into the game while the game is still running for quick iteration.
 
Most hobby engines don't even have one artist on their team, so they have no idea what the content-creation workflow for their first game is going to be... So they create an engine, start making their first game, try to recruit people... and then... try to solve the content workflow issues that suddenly appear.

As well as answering "How do I write gameplay code?", you need to be able to answer "How do I create UI's?" and "How do I import a skinned mesh?", etc... Unfortunately this is often catch-22, as you want to tailor the answers to those questions to the talents and preferences of your content-creators, who you don't have yet.

So if your first game with the engine is going to be a one-man game, then first you've got to learn how to design levels and create art, and then create the engine around your own workflow preferences for those tasks.

 

 

Too bad there are already common formats supported by nearly every single tool tongue.png. Though i don't quite understand the logic of not using more than one format, when said format is usually supported by all systems. Why support Autodesks prioprietary format, when a more common format supported by all 3D tools is Colloda, Obj, and Open Game Exchange? Plus for things like .wav and mp3 files... they are under some hard core licences... so using them in a game engine is no go.

 

But I am going to add in on my previous post. The hardest part of making a game engine, is not to over-engineer. Which I am now simplifying my code down to leaving actor logic updates as single threaded. And representations as threaded. Maybe co-routines will help out when I need to do something like raycasting. Maybe not.

Edited by Tangletail

Share this post


Link to post
Share on other sites



Buster2000, on 17 Oct 2015 - 1:31 PM, said:
Writing an engine itself isn't actually difficult.
It's false, writing a good engine is not easy at all, you have to deal with threading, motion system, cross platform, easy to use, extensible.....

 

 

None of which are difficult they just take monster time ;)

Share this post


Link to post
Share on other sites


None of which are difficult they just take monster time ;)

 

What is difficult in programming then? Unless you are developing a brand new, cutting corners technology, everything in programming can be done given "unlimited" time. Developing a large project for many years in a way that you don't have to ragequit after some time because your codebase became totally unworkable with, can be considered difficult.

So its not just a matter of "I throw code and features at the engine until I have everything I want", but a process that requires planning, knowledge and heavy refactoring unless you have a very good idea of what you are doing. Otherwise you'll just come to a point one day where this feature you want to introduce or this improvement to the workflow will just require you to rewrite pretty much everything, or just start over.

Which could be solved given enough time, ok, but that is pretty much the definition of "difficult" right there.

Share this post


Link to post
Share on other sites
None of which are difficult they just take monster time ;)

Lot of features are easy to implement only because you have source with the paper or a good paper explaining all, without papers on internet, you would not say that.

Edited by Alundra

Share this post


Link to post
Share on other sites

 

None of which are difficult they just take monster time ;)

Lot of features are easy to implement only because you have source with the paper or a good paper explaining all, without papers on internet, you would not say that.

 

But these papers exist therefore I do say that.  The challenge isn't gluing these features together into an engine but writing an interface and tools so that your engine can be reused by other people.  Now you may go on about <insert feature that is only available in cutting edge engines and has very little research done on it> but , that isn't what the OP was asking for.  He didn't mention physics, he didn't mention cross platform, he didn't mention shaders, he didn't mention motion systems, he didn't even mention what programming language he is using.  He just asked for challenges when creating a games engine.

Every single Games Coder I've ever interviewed has had some form of games engine that they have written, but not every coder has written a full game.

Share this post


Link to post
Share on other sites
Every single Games Coder I've ever interviewed has had some form of games engine that they have written, but not every coder has written a full game.

It's a good point to mention, it's better to use an existing engine to make a game and work on an engine when spare time.

Edited by Alundra

Share this post


Link to post
Share on other sites

Will echo what many have already said.

 

The engine runtime is the fun part, and also mostly easy, unless you go for challenging state-of-the-art tech, or try to maximize performance. Creating the scene management, rendering and lighting, physics integration, possible multithreading etc. This can still take a lot of time (easily a man-year) depending on your expertise and how much features you're going to add.

 

After you've got the runtime done, the rest is to make the system usable for actual game creation. Up to this point you probably haven't needed to make any concrete decisions on how the game projects made with the engine are structured, how assets are imported and cooked into a build, how the game logic or rules are inserted and how they interact with the runtime, how the world data is represented for processes like precalculated lighting or navigation data creation, and how to make all the workflows usable for the creators. Now you're going to have to make a lot of decisions which influence what kind of games you can make with the system, and how usable it turns out in the end.

 

It helps if you can handle 3D modelling yourself so you can continuously test from a content creator's point. In reality working on the runtime & tools / workflow will very likely intertwine, I just separated them to illustrate the difference.

 

You can also decide to limit yourself to just creating a coder-oriented runtime library (compare e.g. to Cocos2D or Ogre), rather than a full-blown game engine (like Unity). It will still be a worthwhile learning experience, but probably not something that's directly useful as a game creation tool. Getting to the full-blown stage will certainly take man-years.

Edited by AgentC

Share this post


Link to post
Share on other sites
One challenge I haven't seen any comments on is - depending on the language - is incorporating type reflection and a Metadata system. The reason I bring this up is that it is not simple nor one sided in a C/C++ code base (which I would recommend for an engine - even more so for learning purposes).

The reason I bring this up is because not many have, it offers usefullness, and when incorporated, makes life a breeze when adding new modules. Many have brought up tool chains and the such but have forgotten that to make something - like a great editor, property reflection is extremely handy. For instance it enables easier data binding to your User Interface as well as writing out assets to a file and loading them in later. Other uses is that incorporating reflection, you may get serialization and a network protocol for free. Finally, adding new modules becomes easier if you design an extensible reflection or Metadata system. For instance, incorporating a scripting language becomes simpler when you can look up function and class definitions automatically in code. As opposed to hard coding them to be bound into the scripting language, on top of having to add new bindings with each new class.

Share this post


Link to post
Share on other sites
Yes tool chain is what set apart the licencable engines to the inhouse build engines in the pro studio league. But it is also the support part that crusial. For large production with army of artist. The productivity effeciency is key.

The key here is also the production of a full big game.

But my guess is that not relevant for a hobby game engine starter. Who get it first shot at first engine. For learning purpouse.

But for a engine builder you don't need a full game. Just a demo sized game which incorporates all the features a full game needs.

Then when the small team or lone dev want assets. Our lots of it. There is this alternative way of going PCG way. In pure form the high need for a artist might even disapear. Maybe just one is enough might be luxory. A team where there are more programmers then artist.

A example is crowed funded Star Citezen with big funds doing conventional production also starting multi big studio. Need a productive tool chain which are offered by full licencable engines with source and support. Crytech engine.
Compared to infinity a more pure PCG single dev project relying on PCG procedural content generation.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this