Sign in to follow this  
RobMaddison

Render queue design decision

Recommended Posts

Hi

 

In my entity component-based engine, I have a 'Renderable' component.  This can be added to an entity and establishes it as something that can be rendered. The Renderable component holds smart pointers to all the resources required to render the entity.  In order to sort my render queue, I have the concept of a 'RenderToken' which, at its lowest level, has a material ID associated with it (other things in the 'key' are depth, transparency, etc).

 

If I have an object with two materials, I can render its parts seperately (not necessarily anywhere near each other in the queue).  What I wanted to try and decide is where to store these RenderTokens.  Initially, I had them in a vector in the RenderableComponent but that meant in the RenderToken I had to store a smart pointer back to the Renderable Component (which ends up being cyclic).  The reason for this is that I just pass a vector of (visible) RenderTokens to my rendering engine and in order to make each draw call, the rendering engine needs to reference the Renderable for resources, textures, meshes, etc.

 

This feels wrong, cyclic dependencies feel like a design fault to me, so where should I store my render tokens?  Should they be made into a component themselves?  i.e.:

 

Entity:

   Renderable Component

   Mesh Component

   Skeleton Component

   RenderTokens Component

 

This also feels a bit odd, render tokens feel like they should be lumped in with the renderable component.

 

Any thoughts?

   

Share this post


Link to post
Share on other sites

Why did you have the RenderTokens as a vector in the RenderableComponent? Maybe I misunderstood, but wouldn't one instance of a RenderableComponent just contain one RenderToken, since it won't have multiple materials at the same time?

Share this post


Link to post
Share on other sites

If I have an object with two materials, I can render its parts seperately (not necessarily anywhere near each other in the queue).  What I wanted to try and decide is where to store these RenderTokens.  Initially, I had them in a vector in the RenderableComponent but that meant in the RenderToken I had to store a smart pointer back to the Renderable Component (which ends up being cyclic).  The reason for this is that I just pass a vector of (visible) RenderTokens to my rendering engine and in order to make each draw call, the rendering engine needs to reference the Renderable for resources, textures, meshes, etc.

 

Sounds like you can fix this by tweaking the way your renderer works.  Why not hand off the entire Renderable component to the render which can then ask for the RenderToken vector (via some interface in Renderable) for the purposes of sorting?

Edited by nonoptimalrobot

Share this post


Link to post
Share on other sites

Why did you have the RenderTokens as a vector in the RenderableComponent? Maybe I misunderstood, but wouldn't one instance of a RenderableComponent just contain one RenderToken, since it won't have multiple materials at the same time?


The way it progressed, the RenderableComponent contained pointers to the various resources required for that object (diffuse, normal textures, etc) and it represented the whole renderable aspect of that entity. In order to draw parts of the object in a batching way, each rendertoken represented a material ID and a sortable key as part of that model.

So I had a method on the RenderableComponent (GetRenderTokens) which essentially added the rendertokens on that object a to a vector. After calling that on all objects visible with the same vector, I could sort the entire rendertokens list then pass that list to the renderer. The render then essentially went through this collection and for each said, e.g. 'I need to draw mesh subset 45 which has a material ID of 87' - it then needed to reference the RenderableComponent in order to get the entity's mesh in order to draw it.

Now that I've written it out, it seems a bit convoluted.

Share this post


Link to post
Share on other sites

What's actually doing the sorting? I think it would make more sense to me to pass the renderer an unsorted vector and let it do whatever sorting it needs.

Share this post


Link to post
Share on other sites

What's actually doing the sorting? I think it would make more sense to me to pass the renderer an unsorted vector and let it do whatever sorting it needs.


That bit isn't really the issue, it's just a simple sort which can be called from anywhere. Currently it is actually called early on in the rendering engine.

Share this post


Link to post
Share on other sites

Why did you have the RenderTokens as a vector in the RenderableComponent? Maybe I misunderstood, but wouldn't one instance of a RenderableComponent just contain one RenderToken, since it won't have multiple materials at the same time?


I re-read this and it's starting to make more sense. I think my issue is that my RenderableComponent was supposed to be a characteristic about the entity that says 'this entity is renderable' and my mistake was putting everything in here including the details of the materials (vectors of texture pointers etc).

I've been thinking about it and it makes more sense for a material system (which I haven't started yet) to have all the information about texture resources, etc and for it to set all the necessary textures and renderstates rather than the RenderableComponent (my entity system isn't just data based, my components have functionality too).

With that, all the RenderableComponent would need to hold is the rendertokens (not forgetting that a rendertoken is essentially just a sortable material reference) but the rendering engine still needs access to the mesh so perhaps it makes sense for the rendertokens to be stored within the entity's mesh component?

Share this post


Link to post
Share on other sites

i use the following databases:

 

meshes

textures

materials

models

animations

animation players

entity types

object types

entities

objects

 

an entity has a model, a texture set (for different skin tones), an animation player, and a current animation.

an object has a mesh, texture, and material, or a model, an animation player, and a current animation.

 

models consist of meshes, textures, and materials.

 

i use a struct called a Zdrawinfo to pass drawing information around. It holds primitive type (mesh, model, 2d billboard, or 3d billboard) mesh, texture, material, model, animation player, scale, rotation, translation, cull, alphatest, and clamp information, as well as a world matrix for complex transforms.

 

in my newest code architecture, object types and individual entities has a Zdrawinfo with basic drawing information such as primitive type as well as mesh, texture material, scale, cull, and alpahtest, or what model to use.

 

to draw, i make a copy of the Zdrawinfo for an entity or object, fill in the transform and other info, set the model to the current animation frame and texture set (if used), then send it off to the "drawlist". The drawlist calculates the world matrix for the mesh if needed (IE a complex transform matrix was not supplied, just scale, rotation, and translation values). Then it adds the Zdrawinfo to a list of textures used in the scene. each texture in the list has a list of meshes that use it. Models (rigid body) are broken down into their component meshes and added to the drawlist as separate meshes. When it comes time to draw, the draw list draws all the meshes in order, based on texture. It includes a state manager to filter out redundant state changes.

 

all the databases are implemented as arrays. all references are done using ID numbers, which are the indices of the database arrays. so the first texture in the texture database has a texID of zero, and so on.

 

originally, i kept all the Zdrawinfos in the drawlist in one big list. When a Zdrawinfo was added, its range to the camera was also calculated. when it came time to draw, the list was sorted on texture, then mesh, then far to near.  Later, i developed the 2D list while working on a fast way to draw large chunks of terrain containing thousands of meshes. So i modified the drawlist to use the new 2D list data structure. Right now its so fast, i don't even bother to sort on mesh, material, or near to far. I don't sort at all, although i may go back and add sort on mesh, then material, then near to far.

 

the data in the Zdrawinfos in the drawlist, especially the old single list version that got sorted, is similar to your "rendertokens".

 

in my case the basic drawing info is stored in the entities database or the object types database, which in your case would be a "drawinfo" component in your entities.

 

then you would proceed along the same lines that i do. pass your drawinfo with your pointers in it to your renderer, and let it do its thing. 

 

its possible that using pointers instead of ID numbers and arrays, or some other aspect of C++ specific syntax is causing your difficulties.

 

I see questions like this all the time. Sometimes i wonder how anyone gets anything done with such difficulties.

 

As you've probably surmised by now, i don't have these issues because i don't use C++ syntax for such things.

Edited by Norman Barrows

Share this post


Link to post
Share on other sites

I think my main issue here is ownership.  The RenderTokens collection I want to pass to my renderer needs to be generic in that they could have come from any model/entity/filter and the renderer won't really care.  Its responsibilty is to work through each render token and render it.  For this to happen, the rendertoken needs some kind of link back to the entity it belongs to in order to get the mesh but then it also needs to belong to the entity in some form in the first place, I'm not sure I can see how I can get around the cyclic dependency the way my architecture is currently set up.

 

Coincidentally with Norm, all this has come about because I'm moving my terrain system into my engine (from my sandbox) and with the relatively intricate way I do my terrain layering (which is super-fast and very flexible), I wanted it to go through the same render process, e.g. each terrain layer just becomes a render token so the renderer doesn't need to do anything special when it comes to drawing the terrain.  As far as it is concerned, it just needs a mesh (and subset ID) and a material and that's that.

Share this post


Link to post
Share on other sites


I think my main issue here is ownership.  The RenderTokens collection I want to pass to my renderer needs to be generic in that they could have come from any model/entity/filter and the renderer won't really care.  Its responsibilty is to work through each render token and render it.  For this to happen, the rendertoken needs some kind of link back to the entity it belongs to in order to get the mesh but then it also needs to belong to the entity in some form in the first place, I'm not sure I can see how I can get around the cyclic dependency the way my architecture is currently set up.

 

the link back is required because you don't pass all necessary info in the render token. so you need to link back to the sender to get the rest of the info.  your render token is basically identical to my zdrawinfo struct, except instead of containing all the drawing info, its only contains some, or perhaps just the link back to the entity which contains it all.  there's nothing really wrong with a link back, it is one way to pass a "pointer" to the drawing info. However the more customary way to program is to pass variables to a subroutine, not a pointer to a data structure owned by the caller. OTOH, you could jut think of it as another from of call by reference, which is what it essentially is. So it looks like your choices are to pass all info in your "render token" or by some other means (call by value), or used the link back (call by reference). 

 

I personally find the Zdrawinfo struct to be an invaluable data structure useful for many things. It was originally created just to pass parameters to the drawlist. but it has since evolved into the preferred way to store drawing info of any type, any where. And it makes perfect sense too. Its a perfect example of "let the task define the data structure". what could be more natural than to store all drawing info everywhere in the game in the format the renderer uses?

Share this post


Link to post
Share on other sites

 


I think my main issue here is ownership.  The RenderTokens collection I want to pass to my renderer needs to be generic in that they could have come from any model/entity/filter and the renderer won't really care.  Its responsibilty is to work through each render token and render it.  For this to happen, the rendertoken needs some kind of link back to the entity it belongs to in order to get the mesh but then it also needs to belong to the entity in some form in the first place, I'm not sure I can see how I can get around the cyclic dependency the way my architecture is currently set up.

 

the link back is required because you don't pass all necessary info in the render token. so you need to link back to the sender to get the rest of the info.  your render token is basically identical to my zdrawinfo struct, except instead of containing all the drawing info, its only contains some, or perhaps just the link back to the entity which contains it all.  there's nothing really wrong with a link back, it is one way to pass a "pointer" to the drawing info. However the more customary way to program is to pass variables to a subroutine, not a pointer to a data structure owned by the caller. OTOH, you could jut think of it as another from of call by reference, which is what it essentially is. So it looks like your choices are to pass all info in your "render token" or by some other means (call by value), or used the link back (call by reference). 

 

I personally find the Zdrawinfo struct to be an invaluable data structure useful for many things. It was originally created just to pass parameters to the drawlist. but it has since evolved into the preferred way to store drawing info of any type, any where. And it makes perfect sense too. Its a perfect example of "let the task define the data structure". what could be more natural than to store all drawing info everywhere in the game in the format the renderer uses?

 

Thanks for this, Norm, I'll carry on with it as it is then.  It feels like a pretty neat architecture, I was just conerned about the cylclic dependency but it sounds like it's nothing to worry about.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this