Propagating data through an engine to constant buffers

I have tortured myself over this decision in several of my engines that I've written.

Generally I have a couple of things that I look up. Usually I'll have my materials have some way of knowing what constants are in my shaders then doing a lookup for the size of the constant, register location, etc. Then the gameplay side just knows the name of the constant and provides a void* to the material.

Now the downside is that every shader needs to have additional information to tell the game what constants it is looking for and other information regarding it. This can be a pain, and I just do it by hand.

The main problem you're having is abstracting everything using the component model; it is creating a conflict of what things you should pass to the shader manager in order to set the correct data.

Models are not render components.

A model (at run-time) is a renderable model, and you need to create a specialized library in order to know what things you should pass to the shader—this is the only way of don't mix up game engine high-level architeture with low-level architeture (models are lower level than a render component).

A model library knows what a graphics library is. Thus (the game simulation), a model that can be rendered is just a bunch of buffers and shaders information such textures, a local-matrix, and it has no loading information (once they're loaded you can't change what you're rendering because isn't a animation).

The graphics library (it supplies slots to bind a matrix, a option to set the current view-matrix, projection-matrix, shadow-map matrix, etc.) is what a model and a terrain need to be rendered (they play together).

Terrains aren't models.

A terrain library knows what the graphics library is. A scene doesn't have terrains (a terrain isn't a actor nor a camera nor a light), so you need to re-iterate when you say that a render component can have a pointer to the terrain.

After separating a models and terrains you need a way to pass the information to the shader. That's when you see what things you can reduce in order to not keep the graphics library dependent of any other module.

There is no 100% default shader data.

Beginners graphics programmers trying to abstract 100% a shader is the same thing of physics programmers trying to abstract constraints, and they're only asking for pain (at least at the beginning).

Need some vegetation effects in the game? Create vegetations effects in your shader and make sure that you have a way of your shader manager (at software level) supply that to the class that manages vegetations that above the graphics module—this is just saying "create a simple slot in the shader and a function that passes the data (time, wind characteristics, etc.) to the shader at run-time".

A entity-component system it is at the highest level possible of a engine (if you define a engine being the module that uses everything on low-level modules).

That said, you should be able to create some kind of shader parser in order to give the artists the ability to have a high-level view of your rendering architeture (fog, vegetation parameters, lighting models, shadow parameters, etc.). If you're not using any shader parser you can at least re-define what models, terraints, and game render components are before using them.

IMHO that's one flaw of a entity-component architeture because you keep stuck on "I can't do that, I need a abstraction layer", etc. instead of defining only one thing:

"Use componentization over inheritance, and make sure that you keep everything inside the engine module (the highest-level module on the game engine)".

@Seabolt Thanks for the input. It's definitely a challenging design to get just right :)

@Irlan Also, thank you for your comments. I think you and I are actually on the same page in a lot of places, and maybe something like the shader manager is the missing piece for me.

To clarify a few things that may help the discussion:

Models are not render components.

Correct! Render components own a pointer to a model instance, that's all. A model actually knows nothing about render components or anything in a higher level system, and that's the aim of the post - to keep it that way :)

Your and my concepts of models and graphics libraries and the separations between the two are matched, also. My graphics library has no concept of a model. In fact, it's mostly just a thin wrapper around DirectX 11 functions so hopefully when the time comes, porting to other API backends (OpenGL/DirectX 12) isn't too much of a hassle. The graphics library can be used to create various buffers (vertex, index, constant), textures, device contexts, etc. (you get the picture), and has the ability to set data to the actual pipeline slots, but that's about the extent of it.

A model simply contains a mesh and the data needed to render that mesh (save for the constant buffers). A mesh just contains a vertex buffer, an index buffer, and topology descriptions, but has no knowledge of anything higher level than that.

Terrains aren't models.

True. My terrain class has all sorts of data about the terrain instance, including methods for getting the height at a specific coordinate, etc., that have nothing to do with actually rendering the terrain. But it also necessarily contains a model as described above (with a mesh containing a vertex and index buffer), and that model can be added to an entity as its render component so it will get drawn in the same rendering loop as every other entity.

And that's where the current hiccup in my design is, and possibly something the shader manager will solve for after I've thought it through a little more. A model is as simple as any simple thing can be, so when its any particular model's turn to be rendered (be it a house, a wizard, or a terrain), I need a way upstream of the model to ensure that it's is getting everything it needs for drawing itself. For the most part, it contains this - a model has a mesh which has the vertex and index buffers and topology requirements to send to the graphics context. It's shaders and shader resources are stored as part of its subset descriptions. It's just the dynamically updating things that I'm still trying to get figured in, like a constant buffer that has a view matrix, a camera position, and frustum planes, which depends on the specific frame.

Does that help clarify where I currently am with this?

Thanks,

WFP

But it also necessarily contains a model

Terrain necessarily does not contain a model/mesh. It constructs primitives through basic math for the X and Z, and via a heightmap for the Z. It also has specialized LOD methods, and if you are using GeoClipmap terrain there is a special update required for the heightmap texture itself.

Vegetation necessarily does not contain a model or mesh.
Volumetric fog. Water.


Your problem is that you have designed a system that acts like a funnel. At the end of the day, your system requires a certain type of data in a certain format to create a render, which is a design flaw.
Things can render vastly differently from each other, which is why you always let the objects render themselves. Parts of the pipeline can be shared, such as data for sorting in a render-queue, but at the end of the day it needs to be the actual object that does the render. This allows full flexibility and also solves your problem, since objects will be able to upload to shaders whatever data they need, and only that data.


L. Spiro

But it also necessarily contains a model

...
Volumetric fog. Water.
...

Actually any volumetric effect doesn't really need to contain a model.

Also, I'd like to note that you might be trying to generalize materials possibly into single shader. As long as we will have hacks in computer graphics (which means as long as we are not able to run real time bidirectional path tracer without any noise in output image and generate those images at 100 fps with highly complex BSDF based materials), you shouldn't do that.

Just a little explanation - there is a generalized mathematical model describing how the light scatters around surface (e.g. incoming light vs. outgoing light), which is called BSDF (Bidirectional Scattering Distribution Function) - there are even some generic BSDF already implemented in some offline rendering packages (these materials are most generic, you can describe almost any material using it, and they are terribly slow) - BSDF is a superset of less complicated and faster set of functions - like BRDF for example (I guess you've heard about BRDF already). Using correct, physically based implementation of BRDF is still not possible currently (you need ray tracer for that ~ although I get quite good results with my ray tracer - for simple scenes (Sponza, and such) it even gets realtime on solid hardware)

BSDF is quite far future (for interactive rendering), because I don't see a way how you could work with it inside a rasterizer-based renderer and still produce at least semi-accurate results.

Actually any volumetric effect doesn't really need to contain a model.

Nor does terrain or water or vegetation. That’s why I listed them there together.
Hard to tell if your emphasis is on “any”, which would in that case mean that you are just expanding upon my list.
And indeed, no volumetric effect needs a model/mesh.


L. Spiro

Also, thank you for your comments. I think you and I are actually on the same page in a lot of places, and maybe something like the shader manager is the missing piece for me.

Just remember of the SRP (Single Responsability Principle): if you're doing something that is unrelated to a class you need to create another class and give that this responsibility. So, dividing tasks it can be a great start.

But it also necessarily contains a model as described above (with a mesh containing a vertex and index buffer), and that model can be added to an entity as its render component so it will get drawn in the same rendering loop as every other entity.

The more abstracted you do things that don't necessarely need to be, more you tend to write unmaintainable code. A terrain is so specialized that you find books out there like "how to render terrains", "shader guide for terrains", "pratical rendering with terrains", etc. A terrain doesn't have a mesh. It has buffers, a lot of texture layers (to do texture blending, etc.); do not mess up with the models.

Correct! Render components own a pointer to a model instance, that's all.

Looks like your "render component" is a model instance. If it is then it isn't a problem, but a render component can be anything that can be rendered at the game-side.

If you're not confortable with the entity-component-system I'd recommend creating a simple hierarchy such game entities (used in CryEngine, ID Tech, etc.) and keep sub-dividing the entities responsibility so do the components. I don't use ECS myself because I think that for one single person to manage that in a engine it's asking for unmaintainable code.

Hi all,

Thanks for the additional comments. I think it's starting to become more clear how I'm going to need to restructure a few things, but I have a few more questions that might help me out a little.

But it also necessarily contains a model

This was mainly in the context to how I have my heightmap-based terrain setup. As stated, in my engine a model is just a vertex buffer, index buffer, and a subset table. With my approach, I'm storing all of these in a single model that the terrain class builds during initialization and owns and submitting it to the input assembler as a control patch (D3D11_PRIMITIVE_TOPOLOGY_4_CONTROL_POINT_PATCHLIST). Are you saying instead that the Terrain class itself should just directly own the necessary buffers and subset table? This isn't a huge refactoring effort by any means to pull out of the model class and give to the terrain directly, it just seems like it's duplicating code.

I guess I see two possible approaches emerging. One is to loosen up exactly what a render component owns, and instead have it be more like below:

// anything that a scene needs to render must implement this interface
class IRenderable
{
public:
  void render(GraphicsContext& deviceContext, FrameRenderData& frameRenderData) = 0;
};

class RenderComponent: public BaseComponent
{
public:
  // basic interface
private:
  IRenderable* pRenderable;
};

In this setup, whether I'm using my current simple heightmap terrain or later on adopt geoclipmapping, for example, as long as the terrain class implements IRenderable, it can be used as a render component in the scene and drawn during the main render loop to the GBuffer (or whatever render target it needs). This would also mean that a model class would need to implement the IRenderable interface, and therefore know how to draw itself. I don't hate the idea too much, but it seems like a model should contain data about what's needed to draw itself, but not actually do the drawing. Maybe I'm just trying to over-complicate and over-abstract here, though.

The other approach would be to have specialized renderers for different types of objects. One would exists for models in their current context, and there would also be a specialized one for terrain, water, vegetation, etc. For example:

class IRenderer
{
  void drawScene(GraphicsContext& context, FrameRenderData& renderData) = 0;
};

class ModelRenderer: IRenderer
{
  void drawScene(GraphicsContext& context, FrameRenderData& renderData) override;
};

class TerrainRenderer: IRenderer
{
  void drawScene(GraphicsContext& context, FrameRenderData& renderData) override;
};
// more specialized renderers
...
//
RenderSystem
{
  std::vector<IRenderer*> m_renderers;
  // scenes can add only the renderers they need
  void addRenderer(IRenderer* pRenderer);
};

With this setup, the renderers themselves are specialized and know only how to draw objects that "match" them. So for example, if I were to later down the road create a new terrain class based on geoclipmapping, I would create a specialized renderer class for it and register that with the render system during scene setup. Regardless of whether or not it has a model, the specialized renderer will know enough about whatever it is it's supposed to be drawing to set the appropriate data to its constant buffers and get it into the GBuffer or whatever other render target it may happen to need.

Does this make sense, or am I still missing it?

Thanks for your help, all.

My approach is a bit different in that low-level rendering is not directly related to an entity.

Low-level rendering means to execute the stages in a graphic pipeline. Each stage has its specific task within the pipeline. A stage that uses the GPU requires a GPU program. From all available GPU programs left after filtering by the platform and user settings, at most a few of them are suitable for a specific stage because they implement the solution to the stage's task.

A renderer as a component of an entity (be it a game object in ECS or an entity in its own) specifies which one of the remaining GPU programs is to be used. In the end it is the interface of the GPU program that need to be fulfilled, e.g. to set all constant blocks as expected and to stream in all vertex attributes as expected. Some of this stuff is provided by the entity via its renderer, probably stored in some other components. Some is from elsewhere, e.g. the camera or the view settings from the viewing system or stage specific settings from the stage.

Collecting this stuff is the task of the stage when building the rendering jobs. In this sense a renderer component is not an active component. Maybe that this gives a bit of flexibility away (although I have not hit such a problem yet), but it relates the responsibility of what to do within stage processing to the stage itself.

Are you saying instead that the Terrain class itself should just directly own the necessary buffers and subset table? This isn't a huge refactoring effort by any means to pull out of the model class and give to the terrain directly, it just seems like it's duplicating code.

A terrain should not be rendered in the same way of a model. A terrain also does not has sub-sets; it contains what a terrain needs to have: vertex buffers, index buffers, textures, etc.—specific functionality.

For more that the terrain graphics data looks like the model data (buffers and resources) you don't need to explicity let a terrain be dependent of a model.

Further, you may want to do specific things with a terrain such terrain rendering, etc. Remove any dependency that you're having with the model and the terrain and you'll be doing in the correct way. This is not duplicated code (since the objects characteristics are totally unrelated and well defined).

Don't be afraid of expanding 10 classes to 100 when it make sense.

Later, you will see that complexity depends of the pre-defined requisites, and in your case the requisites are: terrain, models, and your entity-component system, and you should separate that until no dependency is noticed.

A terrain knows how to render itself, so you can cut-off your TerrainRenderer for a while.

A model knows how to render itself, so you can cut-off your ModelRenderer for a while.

Your "Render System" it is a scene. Yet you said that on the comments but since you're limited to call everything a "system" you're limited also to create objects that can be managed by that system you didn't change the class name and its characteristics .

A scene knows what it contains, so you can cut-off the RenderSystem for a while.

The final result would be something like that (other objects such entites, actors are removed from the example for simplicity purposes):

class CScene {
protected :
     CVector<CRenderableModel*> m_vRenderableModels;
     CTerrainManager m_tmTerrainManager; //It has a lot of terrains, and terrains has specific functionalities of terrains.
};

Simple as that!

Now you know that you can call CRenderableModel.Render() and CTerrain.Render(), etc. in the game-engine side at any time you want. Just make sure you've loaded the resource first .

@haegarr and @Irlan

Thanks for the further explanations. I'm definitely starting to see now what you all mean about terrain not having a model (as defined in my overall structure), but instead just having the very specific data it needs. It also makes it more clear how things like volumetric effects would fit in (as you all mentioned above - similar things that don't have a model). That's helping to clarify a lot and I think with a little bit of restructuring and refactoring I can get this thing a bit more manageable.

I'd like to leave the thread open for a while longer to allow a few others to reply and gain as much perspective as I can before I start the refactoring effort, but again - thanks to all who have contributed!

I'm curious about the approaches listed here.

So if we make it to where each object is able to render itself, it makes threading your rendering a lot more difficult because now the render components need to be thread safe, and it also puts the burden of knowing how to draw on the game sim side.

Do you guys just accept that cost, or is there something I'm overlooking?

I'm curious about the approaches listed here.

So if we make it to where each object is able to render itself, it makes threading your rendering a lot more difficult because now the render components need to be thread safe, and it also puts the burden of knowing how to draw on the game sim side.

Err ... no. The approach I've mentioned does not do so. The render component contributes information of how to render in form of selection criteria, and provides access to other rendering related data, but rendering is executed by the graphic pipeline stages. Further, the stages generate rendering jobs which are enqueued for execution by the low-level rendering backend (the well known "order your draw call" thing).

Oh I see what you're saying. So instead of sending down the components, (mesh, materials), you add a render to a higher level that will send down more specific information like (vert buffers, index buffers, textures, shader constants), and then allow the render system to process those elements?

Hi again,

Glad to see the conversation is still going on. I think I'm really getting things hammered out now, but just wanted to go ahead and post a few more questions and see how they should fit into the new design.

I've just finished reading this post by L. Spiro http://lspiroengine.com/?p=96 and that combined with the thread above has gotten things pretty much cleared up. The last few questions I have are below.

1) My rendering pipeline is structured such that it is based on a deferred shading renderer and backed by a forward renderer for transparent objects. The issue I see here is for objects like, for example, a house that is mostly opaque but may have semi-transparent windows. When I'm building out the render queue, should I just make sure that any subset of an object that is transparent is added to a special post-gbuffer / delayed render queue? Or is there a better way to handle these cases?

2) Does shadow mapping fall under the same overall advice as earlier in the thread? Obviously, most objects simply need a super simple vertex shader to generate their data, but there's also the case for objects with alpha clip and/or tesselation that will need their own methods of getting to the shadow map. In addition to rendering itself, should a model/terrain/object of whatever type also know how to draw itself into a shadow map?

I think with those two items buttoned up I'll be set to knock this out.

Thanks,

WFP

Oh I see what you're saying. So instead of sending down the components, (mesh, materials), you add a render to a higher level that will send down more specific information like (vert buffers, index buffers, textures, shader constants), and then allow the render system to process those elements?

Close to, but not exactly. In fact, a renderer component is not an active component, i.e. it does not send something. Instead, the pipeline stage requests data from the renderer component. This is so because the structure of the pipeline (does it forward or deferred shading? can that stage be processed in a single pass on the given GPU or not?) is not a problem of the renderer component. The renderer component marks an entity as being renderable, denotes a graphic technique to use (e.g. physical lighting) as a criterion for the pipeline stages to pick the correct shaders, and grants an access point for the stages to get vertex data and constant blocks.

The following is related to the process of rendering as an example of what happens in detail behind the scenes. But it is not essential to the approach I've mentioned. You are welcome to read on, but notice that it might be more details than you want to know ;) As a background information, the engine is extremely data driven. Even the graphic pipeline is. So ...

A stage in the pipeline is responsible for visibility culling. It iterates the pool of provided renderer components (this is typically the total set of render components of the scene, but may be already filtered by a previously executed stage, e.g. by object layering). Any render component passing this stage is meant to be rendered. The following stage will apply some shader script. So it asks the render component for the technique to use. The result is used to determine the shader program that implements that technique (with respect to the functionality of the current stage, of course).

Let's use the vertex data as an example of what happens further. The shader program provides an interface declaration, from which a VertexInterface may be part of. A VertexInterface is a declaration what vertex attributes are expected at what slots, i.e. it describes the vertex input assembly in some way. Because a VertexInterface is attached to the shader program, the program obviously expects vertex data to be streamed in. So the pipeline stage requests a VertexConfiguration from the render component, where the VertexConfiguration has to met the given VertexInterface.

Ugh, what? To understand this, one need to know that a vertex source (be it a model's mesh, mesh generator, sprite batcher, or whatever) provides a VertexSignature. That is a description of which groups of vertex attributes exists, where a group is an interleaved structure of attributes with a common update frequency (immutable, static, transient, temporary are examples from the literature), and a attribute is basically an enum-value for a semantic and an enum-value for the data type. Now, using such a VertexSignature together with a wanted vertex capacity allows to allocate buffers from a graphics device. Because more than one buffer may be needed for a given VertexSignature (namely due to different update frequencies), a new structure is build which is named VertexCompilation. It says basically how many vertices with a defined set of attributes of defined data types can be stored by using which buffers. Notice that another vertex source with the same VertexSignature is allowed to use the same VertexCompilation, which effectively means vertex batching.

For now, we have a VertexCompilation which is a total set of vertex attribute data available from a vertex source. However, a specific shader program needs a subset of this (not necessarily a real subset, of course) and the attributes streamed into specific vertex inputs. So for every unique VertexInterface used in a graphic pipeline when using a VertexCompilation, a match of the VertexInterface with the VertexCompilation is done and remembered as so-called VertexConfiguration.

Well, all this VertexSomething things are structured blobs, so to say. They are addressed using indices; no pointers or something. And they are stored once they are created or read in as resource descriptors. That's make their use very efficient.

When the pipeline stage has requested a VertexConfiguration from the render component, and got an index number as result, it puts this into a graphic rendering job as state parameter like "activate vertex configuration #5". When the low-level graphic renderer interprets this, it makes a look up in its table of vertex configurations and finds a, say we're running OpenGL, name of a VAO associated with the index. If not, then the original VertexConfiguration is fetched by its index from the graphics support, and a new VAO is build.

Okay let me make sure I've got this, you generate a list of RenderComponents, pass it to the RenderSystem, which when running it's rendering pass will query each component to determine whatever relevant information for each stage?

But then you run into the issue that your RenderComponents have to be locked when being processed by the RenderSystem, (assuming that there is the potential for multi-threaded rendering), where in my experience the RenderComponent is used as the interface between the Game and the RenderSystem.

Is that right?

But then you run into the issue that your RenderComponents have to be locked when being processed by the RenderSystem, (assuming that there is the potential for multi-threaded rendering), where in my experience the RenderComponent is used as the interface between the Game and the RenderSystem.

With fine grained multi-threading: The pool of render components is a view onto the scene. If you want to make e.g. the visibility culling multi-threaded, then partition the pool into non-overlapping sub-pools and give each thread such a sub-pool. Then there is no need for locking. A stage that generates graphic jobs is different in that a common queue of jobs is wanted. So locking has to happen on writing a job into the queue, or else copy sections of sorting IDs together later on.

With "system based" multi-threading: In fact, the low-level renderer in my system runs in an own thread. Rendering is a multi-tier system. The mid-level rendering requests a GraphicFrameContext from the graphic device. The GraphicFrameContext is triple buffered. The request is blocked if the low-level rendering thread is two frames behind, otherwise a context is available and the mid-level rendering continues. Part of the context is a graphic jobs queue which is hence also triple buffered. So blocking may happen when the mid-level requests a context or commits a context for the low-level, or the low-level requests a context or grants a context for re-use. This happens with the frame rate, obviously. Also no need to lock render components, because the components play no role as soon as the graphic jobs are generated.

Interesting approach. I've written a couple of multi-threaded renderers, and they way I avoided duplicate render components was to create my own command buffer for communication between the game side and the low level renderer. So I would batch commands by draw calls, creating commands like SetVertexInput, SetRenderState, SetTexture, SetShader, SetShaderConstants, etc and send down those commands.

I liked this approach because I could make the game side of the rendering system completely platform agnostic, and the renderer doesn't know anything about the game and can be platform independent.

But you're absolutely right that this pattern makes it difficult to alter the renderer functionality.

I might try to make a combination of the two where the game side renderer process the list and then sends the command to the low level renderer.

Sorry I'm somewhat on a tangent, but I like discussing strategies