Jump to content

  • Log In with Google      Sign In   
  • Create Account


Member Since 04 Jul 2011
Offline Last Active Today, 08:04 PM

#5308469 Opengl ECS performance problem

Posted by on 29 August 2016 - 08:12 AM

Sounds good. Basically you want to separate things that are resources like vertex/index buffers, textures, shaders from your components. Maybe when creating new object ask some resource cache for needed resources and provide them to components. Important thing is to not do it for every object separately but keep one loaded version of each resource and just refer to it. 


This can be too much for now, but another optimisation is keeping a state cache. If you render 10 meshes and all of them use the same VAO don't set this VAO 10 times, but keep a state of currentVAO/currentShader/currentRenderState and check if you really need to make GL call. This allows to grealy save on amount of GL calls, and more so if you queue your geometry so you can sort it based on shader/material/state to achieve even less state changes. 

#5308465 Opengl ECS performance problem

Posted by on 29 August 2016 - 07:32 AM

Well, it's hard to say for us what InitSystem does in a big picture (so far we know it allocates a lot of resources) and how often the update is called. But if you're doing this loop for every created bullet object that should be very lightweight (check flyweight pattern for example) and creating separate buffers, shaders and so on - then it's already pretty bad for performance, doesn't have to happen each frame (and for bullets that are short-lived objects it may happen quite often). 


Considering you have bullets and fighter objects, looking at some basic situation you should have no need to use more than 2 different shaders (like one for solid static objects and other for some billboarding or whatever technique you use to render bullets, I don't even know if this is 3D or 2D case).


You're correct about VAO, you can just bind VAO and draw all same objects with it, changing only model matrix. 

#5308461 Opengl ECS performance problem

Posted by on 29 August 2016 - 06:46 AM

You're correct that you're wrong :) you can (and should) reuse the buffers (and everything else too). In rare cases where geometry is so dynamic it has to be changed each frame you can do it, but not in a way you're doing it in your code, but rather mapped buffers and not initializing every GL resource each update call. Allocating objects in such loops each frame without any sort of pooling isn't helping either.


First thing I would suggest is getting rid of re-updating GL resources each frame.

#5307340 Abstract Classes and returning a varying Type

Posted by on 23 August 2016 - 03:24 AM

Thanks a lot, everyone : )
Noizex, I understood the implementation but the key-identifier part. While I understood that std::strings can be a bit of a power consuming thing to compare, how would I do it with integers?
Well, example:
A game object gets created, it knows a file path of its texture. Now it requests the asset-manager if this has been allocated already. The asset-manager would iterate through a list of collected assets with all their <type>s.
What I thought about is to compare the strings with each other, as every asset-item knows its original path. Storing std::strings is not that lovely for such an undertaking, especially comparing them. If I would use something like you suggested, I could simply compare numbers, is that right?

The new game object would simply send the type it is looking for together with an ID, maybe send the file-path as well, to directly allocate if it does not exist.
But how would I define what asset would get what number? Or is there something better?


I think you're confusing several concepts here and complicate unnecessarily. When you load the resource you want to supply whatever ID you need to use to identify this resource and its version on a medium that you store it. It may be file path, it may be just a name or numeric id, it may be something more complex like a struct that contains both, filename and version. It doesn't matter what type it will be, as long as your loader knows how to deal with it. One thing matters - type should be hashable because you want to store references to your resources in a unique_map or similar container for fast lookup. You don't want iterate over a vector and look up for the resource comparing strings, as you suggest above.


So for resource key just use whatever type is most convenient for you. Strings are perfectly fine for this, as they're hashable. You can make it all compile-time even with some C++ magic, but that's probably only needed if you do a lot of lookups on thousands of resources.

In my case, I don't ask manager each time I need the resource (so, each frame just before rendering), I use a simple refcount and don't return plain pointer but a handle that can be resolved to a pointer (this also acts as a proxy and allows returning placeholder when doing async load). This resolving doesn't use a map but rather a vector, and a handle keeps index to the resource in that vector (this is the main container for my resources, map just holds references in case the key-lookup is needed through Get<T> method). This requires designing a good policy of when the handle is valid, what happens when resources in vector need to be moved to other indices and so on, but otherwise is fast where it's needed and allows easy lookup when it's needed.


In terms of code it looks a bit like this snippet:

struct Model {
   Resource<Shader> shader; 
   Resource<Texture> texture;
   Resource<Mesh> mesh;

   void LoadResources(ResourceManager& resources)
      # Here it does the std::unique_map lookup on the Store class (mind that Store class is provided by default
      # but also can be user-defined with completly different storage implementation, so this unique_map is just 
      # what is the usual case)

      shader = resources.Get<Shader>(ShaderKey("base_shader", E_NormalMapped | E_Lighting));
      texture = resources.Get<Texture>("texture.dds");
      mesh = resource.Get<Mesh>("mesh.iqm");

Now a bit on what happens when you Get<T>:

  1. Find appropriate meta data for resource type T - meta data contains load/store/retrieve function callbacks, Store object and Loader object
  2. Call Store->Retrieve(key) to check if resource exists in the cache
    1. If resource exists, Store returns Resource<T> handle to it and increases refcount
    2. If resource doesn't exist Store returns empty handle
  3. If handle is empty, proceed to loading the object, so Loader->Load(key) is called
    1. Depending on the loader, it either fetches resource from somewhere, or creates it using factory class, this all happens inside Loader and other classes like Store or ResourceManager itself don't know anything about it. Loader just has to return the resource (this time it's unique_ptr) or nullptr
  4. When loader returns and the handle is not empty, it gets stored into Store, increasing refcount to 1 (so the store callback is called with the unique_ptr, which is then moved into the Store, as the store is the owning entity for resources)
  5. Handle is returned to user

Later on, when I actually need to use the resource I do it as if it were a normal pointer to resource it holds:

RenderModel(mesh->GetVertexBuffer(), mesh->GetIndexBuffer(), texture->GetId(), shader.get());

When -> is called, the handle does the call on the owning Store (to which it holds a pointer) like:


where m_ResourceIndex is index into resource vector. This way it can return placeholder, or do all sort of things, because it has this indirection which you don't get if you point straight to Resource* pointer.


So to rendering system I usually pass straight numbers of resources (GLuints in my case), but sometimes I need resource object itself like in Shader case (I need to call few more methods and Shader keeps the state for this, like caching uniforms per shader type). This is why last argument is passed as shader.get(), which returns plain pointer to Shader*, just like std::unique_ptr and similar smart pointers. There is a policy that resource manager will never free any resource during rendering phase, so passing raw pointer and holding to it for a while (during the journey of RenderTask through queues, sorting, and into actual render call) is fine because we guarantee that the resource will stay alive during that time.


It's still a naive implementation so there may be some problems, but it worked for me so far and it's quite flexible so I hope it helps you a bit and not adds more confusion :)

#5307198 Abstract Classes and returning a varying Type

Posted by on 22 August 2016 - 08:28 AM

You may consider making it a bit more generic - with some template programming you could have resources of any type and not coupled with some common base class which is not really a good solution (like deriving from Resource class or similar). Not only it will have broader use (for example you could use some external classes that you can't derive from common base without modifying some library), but also should be faster because you won't use virtual calls for something so crucial as getting resource id, something that will happen a lot each frame.


Think of a handle like Resource<Texture>, Resource<Shader>, Resource<Config> etc. You could use resources that don't necessarily use the same key to fetch them (so for example instead of std::string some more sophisticated struct or maybe uint?). 

This way you don't have the problem you seem to describe - that you have several "semi similar" classes, but they all have their own quirks and data. If you have Resource<Texture> you know the object you're dealing with is a texture. If you have Resource<Shader> you know it's a shader and may have completly different interface than texture. You are tying your hands with a common interface for things that may be completly unrelated in terms of their functionality, and only share one thing - that they're loaded and managed in a way that minimizes duplicates. 


In terms of loading, that's where you specify what resource you're querying:


auto texture = resources.Get<Texture>("mytexture");

auto shader = resources.Get<Shader>(ShaderKey("myshader", shaderBitMask));

auto framebuffer = resources.Get<Framebuffer>(2); 



#5306509 Naive deffered pipeline

Posted by on 18 August 2016 - 04:29 AM

Exactly, you do one geometry pass (naively, not mentioning special cases here) and write to several gbuffers (textures) at once. I suggest reading some tutorials about deferred rendering to get a better grasp on this concept, there are several out there, even specifically about OpenGL:




#5303070 Game Engine Advice

Posted by on 29 July 2016 - 04:34 AM

Here is a good book on game engines, written by one of Naughty Dog's programmers: http://www.gameenginebook.com/

I'm afraid that I will unRecommend that book.I have to admit that a havent read it(Just a few topics here and there). The book contains a lot of trivially google-able knowledge that is applicable to amolst any type of programming, sure those skill are 100% a must, but the book won't eduacate you anything about the large topics that are really hard and needed (editor and all that goodness around it, how does it blend with the game world, undo/redo, asset management, different approches to different game types, ect..) Correct me if I'm wrong.


Wait, you haven't read the whole book and you say you can't recommend it? Most of the knowledge these days is google-able (especially parts or whole books...). This book contains enormous amount of know-how regarding advanced game engine programming, tips coming from real-life situations, author's comments and so on. I own it myself (latest edition) and would recommend it to anyone who wants to write anything bigger than Pong or Pac-man game (and even those kind of games would benefit from concepts described in this book). I'm in no position to say you can't write what you wrote, but your post would be way better if you skipped that first paragraph...


I will second opinions from other posters, the books I myself own and can recommend, as I read them at least once:

- Game Engine Architecture 2nd edition by Jason Gregory

- Game Programming Patterns by Robert Nystrom

- Game Coding Complete, 4th edition by Mike McShaffry


These are 3 solid books about game engine and programming patterns useful in game dev. Other books I could recommend are more focused on certain topics, like math/real-time rendering/physics/AI, but the question was more about general engine itself, so these three above are good introduction.

#5302631 Rpg Stats - Temporary Changes, Harder Than I Realised!?

Posted by on 26 July 2016 - 08:11 AM

Looks nice, but if the main point of Effects is to apply (and de-apply) StatModifiers, why not just have Effects contain a list of Stat/value pairs, so you don't have to implement start_effect and remove_effect each time you define a new Effect?


Main point of effects is to give total flexibility on implementing various effects that can affect character - permanent curses, diseases, poisons, buffs and debuffs, spells - so these functions are just hooks that writer of effect uses for a starting/ending point of an effect. This goes way beyond stat modifiers, so I wanted to keep stats and effects separated, because some effects won't even touch stats but change appearance of character, modify it's behavior and so on. The effect could for example do this:


   void start_effect()


       print("You slowly turn into a frog!");

       old_model = target->get_player_model();




   void remove_effect()


       print("You turn back into " + old_model->get_race() + "!");




Though I guess it's a good idea for a base effect that only affects stats :) So instead of creating separate Effect classes I could write StatChangeEffect() and just pass array of stats to affect, without the need to care about adding/removing modifiers for each one. But the general idea is for the effect class to be as flexible as possible in terms of how it affects target.

#5302626 Rpg Stats - Temporary Changes, Harder Than I Realised!?

Posted by on 26 July 2016 - 07:46 AM

In my implementation I have two layers - stat modifiers and effects. Stat modifier may be caused by effect, but effects can do much much more (anything really, from various disease effects, to just lowering certain stats). So the idea is that effect has start_effect() and remove_effect() methods which in case of pure stat modifiers are the ones responsible for adding and removing them. So it looks a bit like this (some Angelscript pseudocode):


class NastyDisease: Effect

    StatModifier @str_mod, @vit_mod;


    void start_effect(object target)


        @str_mod = target->add_stat_modifier(Stat::Strength, -20);

        @vit_mod = target->add_stat_modifier(Stat::Vitality, -30);  



    // Called when effect expires

    void remove_effect(object target)







This makes StatModifier class just a dumb storage for how much and what stat is modified, and the expiration is handled by Effect class (on a usual timer basis, I don't check upon requsting stat that I should remove some modifier, if the effect expires appropriate function is called and it removes these modifiers.


There is slight problem with this, because if something goes wrong in remove_effect() function (runtime error), it may keep the effects on the player, but I guess if that happens something is really wrong. 

#5302602 Rpg Stats - Temporary Changes, Harder Than I Realised!?

Posted by on 26 July 2016 - 03:19 AM

You can cache the result once per frame but it should be cheap by just ask it each time. In this way you avoid all kinds of bugs, note that even Morrowind and Oblivion had tons of bugs related to temporary stats, so I would go the "slow" but safe way, and later optimize only if needed.


If you have a single point where buff is added and removed, why not keep a dirty flag and recompute only when it's set to true? If buff can't be changed outside of those two methods, there is no reason why the value would change from one call to another. 


I would remove any logic governing lifetime of an effect from this function though, just iterate over array of effects, set current value, clear dirty flag. If any effect is added or removed, set dirty flag, which forces stat to be recalculated. I just am not sure that function returning strength should care whether the effect should be removed or not, it just goes over effects in the array and applies them. Other part of code guards the lifetime and removes the modifier from the array at appropriate time. 

#5301389 Possible Bug In Asrun Example

Posted by on 19 July 2016 - 12:08 PM

As I played with various AS concepts in a small program that I based on "asrun" sample I started getting segfaults, which I tracked to be caused by adding a simple destructor to a class that was instantiated, called some method that caused exception to be set and then exited. 


Took me a while, but I think I found the problem and it's probably worth fixing. This example uses callbacks to request and return context from/to a global context pool.


#0  0x000000000044f666 in asCContext::SetObject(void*) ()

#1  0x00000000004950d8 in asCScriptObject::CallDestructor() ()

#2  0x0000000000494ef6 in asCScriptObject::Release() const ()

#3  0x000000000047ae27 in asCScriptEngine::CallObjectMethod(void*, asSSystemFunctionInterface*, asCScriptFunction*) const ()

#4  0x000000000047ad07 in asCScriptEngine::CallObjectMethod(void*, int) const ()

#5  0x000000000045b059 in asCContext::CleanStackFrame() ()

#6  0x0000000000459f8c in asCContext::CleanStack() ()

#7  0x000000000044edac in asCContext::Unprepare() ()

#8  0x000000000040779a in ReturnContextCallback(asIScriptEngine*, asIScriptContext*, void*) ()

#9  0x000000000046cf8b in asCScriptEngine::ReturnContext(asIScriptContext*) ()

#10 0x000000000044a21b in CContextMgr::DoneWithContext(asIScriptContext*) ()

#11 0x000000000040718d in ExecuteScript(asIScriptEngine*, char const*, bool) ()


Key thing probably is exception and destructor and how ReturnContextCallback function returns context to the pool. Here is the function:

void ReturnContextCallback(asIScriptEngine *engine, asIScriptContext *ctx, void * /*param*/)
	// Unprepare the context to free any objects it may still hold (e.g. return value)

Seems like the context is first returned to the pool as "ready to use", and only then unprepared. In my situation (after exception thrown) this causes context to be cleared of objects, which causes that destructor to be called. To call destructor, it has to fetch context, which it does through callback "ReturnContextCallback" and it yields the very same context as the one that's being unprepared. 


I think I fixed this by just moving ctx->Unprepare() above the line that pushes the context back to the pool.


The script I used to reproduce this is simple, it just has to throw exception at some point, while also having a destructor.

class Player
   void foo()
      int x = 0;
      int y = 1 / x;

int main()
   Player player();
   return 1;

Btw, is this correct place to fill some possible bugs/requests or there is some system for this? 

#5297668 How to import animations from commercial models (FBX, X) into Blender

Posted by on 23 June 2016 - 02:10 AM

Thanks for all replies. Turned out that there was some problem with FBX version probably, plus that file I bought was far from well organized. I upgraded Blender to 2.77 and used this tutorial:




and it finally worked, so I'm posting it, maybe it will help someone. Also had weird problem where everything imported but not the armature of a model, after long research and digging through FBX import addon (crazy code, thousands of lines) I managed to pin it down to a loose root bone that was encountered before the real armature root bone, and it caused only that one wrong armature to be imported to Blender. I skipped the bad one by dirty hack in addon code and suddenly the correct armature appeard. I filled bug and I'm waiting for some reply from addon's maintainer so hopefully this will be resolved for all cases.

#5268966 RakNet still alive?

Posted by on 03 January 2016 - 05:07 AM


there is supposedly a ton of buffer overflow bugs


Don't believe everything you hear on the internet, unless there is verifiable evidence.
Can you link to one active bug report about a buffer overflow bug in RakNet (that shows a problem in the code)?

Second, if you need reliability over UDP, use Enet.
If you are using a particular game engine, like Unity or Unreal, then look at the default solutions for those engines.



Hence "supposedly" smile.png I just got that impression by looking at issues on github, and haven't really checked&confirmed any of those (I just need library that works, if I were to spend time checking for these things in every thing I use I would never get past main() in my project):








Some are not very detailed, but one can get the impression there are more such bugs than should be in mature library. Saying this, I'm still hoping RakNet is stable enough to be used in my project, I already integrated it and it worked in development settings for these years, but these signs (selling RakNet, new owner just letting it go on GitHub where noone even approves PR) made me a little worried. Every time I upgraded to new RakNet version in the past I struggled with issues like errors/warnings, problems with CMake generation, so every time it left a feeling that lib may not be as polished as other 3rd party things i use.


I considered Enet but at the time I made the decision it was slightly too low-level for my present knowledge and what I needed - reliable UDP client-server architecture, but also some helper methods for packing data specific to games (vectors, quaternions), so some nice way to read/write data into packets (like RakNet's BitStream), way to prioritize packets, channels etc. This means that right now I'm probably using 5% of what RakNet has to offer, and probably won't be using much more - I will handle object sync myself, not using some ReplicaMaster, same for chats and all other in-game activity. I may need some secure crypted data connection for logging too (I think RakNet also has it).


I'm staring to wonder what would be better in such case:


1) staying with RakNet, as it works, even if I'm using just a small percent of functionality and there are previously mentioned issues with the current state of project

2) trying to convert to some lower level lib like Enet, and adding needed functionality that I'd lack compared to RakNet (plus making it more in object oriented manner). Are there any libraries based on Enet that are slightly more high-level, but not as big as RakNet?

#5268805 RakNet still alive?

Posted by on 02 January 2016 - 04:00 AM

RakNet is one of the most mature pieces of middleware around. I wouldn't be that worried that the pace of development has slowed.


Besides, Unity uses RakNet, so on that alone, I doubt it is going to die off anytime soon - if there's any danger of it, someone will fork the codebase.


Sorry for resurrecting this thread, but recently I had to rebuild my project from scratch and found out RakNet is no longer "alive". I'm not sure how this is possible. I've been using RakNet for 3 years now, and to be honest it always seemed like really messy framework (but it worked so far with my limited needs). Build files are terrible and always require some hand-tweaking, and there is supposedly a ton of buffer overflow bugs and few other major problems. Source has been abandoned by OculusVR and only some life can be found in forks, which have their own problems and don't compile out of the box on popular compilers like MSVC2015. 


How library that has been around for so many years can end like this? Are there really no alternatives? Should I stick to RakNet since I already use it in my project and try to somehow fix the problems and follow some pull requests from the community, or there is some reasonable alternative (in terms of functionality, I need some UDP network layer with reliability options for client-server architecture). 

#5207386 Standard structure of a large scale game

Posted by on 29 January 2015 - 04:12 AM

That's a question I was going to ask many times myself but can't even phrase it clear enough. I started with small project but now it's grown, has over 300+ classes and client-server architecture. More and more I'm finding myself pondering how to approach something in terms of data-object interaction and it takes many hours to figure some things. For example, I have network synchronized modifiable terrain. I needed it to synchronize when player moves and changes grid position, but I didn't want to clutter player movement logic (which is tricky in itself, as it's over network with authoritative server) so I created some TerrainDb class that receives network messages and deals with response and updating terrain caches. What is my major problem is relation between different systems and classes. Then it turns out such class needs some networking access to receive/send data to server, requires access to World/Terrain and so on. And it starts to get messy already.


Top game logic is handled by GameState, each of which has access to several main objects (like Connection, ScriptingEngine etc.) but most of these states are simple (Login, LoadingScreen, Character menu) until I get to "gameplay" state. This is where so many things need to happen, so many systems need to be initialized and propagated that I have no idea how to design these relations. Should I initialize them all in state's Init() func, making it 100-200 lines long? Pass dependencies to constructors? So far I avoided singletons and I have not a single one, but the net of class relationships becomes very hard to maintain. 


While I'm not making an "MMO" but way smaller networked "sandboxy" game, I checked Ryzom/Planeshift sources and several others that had sources available. They all seem to be just as messed and without deep knowledge you have no idea what's going and and can't even track relations. So many weird classes initialized in weird places, passed around. Is that what every project ends up as? I admit I saw WarZ source code too and was terrified - 200kB classes that do every freaking thing from rendering to physics all messed up and mixed, not even divided into sections of similar theme. 


Mind that I'm talking about lower level and unique things - terrain, networking, scripting etc. I don't think this can and should be done as components (terrain itself is really unique, which would lead to a special case game object that has really unique components and has systems that work just on that one object which seems silly). Component systems seems more like game logic where you fit all pieces together, but this requires some bigger things to exist first. I don't really think I could fit everything in Component-System manner. 


So anyone has any pointers where such design could be learned? Is it just experience? Everyone starts talking about game loop and game states but these things are nothing compared to vast net of systems/services/"managers" required for a more complex game.