Creating a Good Design To Sort, Batch and Draw Renderables

Started by
10 comments, last by Bakura 14 years, 7 months ago
I'm trying to design a renderer that has the ability to batch all the renderables, sort by texture, shader and other parameters and then draw them all at once. Initially, I was thinking of some simple, naive design like this. Using something like that, all renderables are given to a list which sorts them appropriately and then passes them to a renderer which draws them. Some example code would be:

void DrawScene()
{
    renderList.Add(playerRenderable);
    renderer.Update(renderList);
    renderer.Draw();
}

void Renderer::Update(boost::shared_ptr<IRenderList> list)
{
    // Create VBO and pass every renderable's geometry to it
}
void Renderer::Draw()
{
    // Of course, for-each loop isn't in the current standard, but still :P
    for (IRenderable* r : list)
    {
        // Perform necessery transformations for this renderable

        // Set texture/shader

        // Draw this renderable's geometry using correct indices into the VBO
    }
}


I have a few questions: 1) If IRenderList takes IRenderables, then how it will know if each renderable has a texture or shader and be able to sort them correctly? For this to happen, the IRenderables must all share some of the same properties, that is, they all need to contain textures or shaders. This makes the design less flexible. Should every renderable have a shader/texture ID and if a specific renderable doesn't use them, they can just return 0 as the ID? 2) When it comes to drawing everything, how will the renderer know if the renderable needs to transformed or not? And how will handle hierarchical transformations? For example, if there's a person with a weapon, the renderer will need to transform the player and then the weapon. This means that it will have to draw the weapon straight after the player, but what if they're not next to each other in the new list sorted by IRenderList? Should transformations not even be handled by the renderer? 3) Should this system know if the renderable is visible and don't draw it if it's not? I'm guessing this part would be done when adding the renderable to the list, meaning it's not really part of the renderer per se. For example:

if (someClassDedicatedToVisibilityTesting.IsVisible(renderable))
{
    renderList.Add(renderable);
}


Phew, that's pretty much it. I have a feeling I'm going about this the wrong way. As you can probably tell, I haven't done anything like this before, so I'm grateful for any help and advice so I don't fall into as many problems later on. :)
Advertisement
I can't really provide a critique of your method, as you haven't really fleshed it out enough to be meaningful. I will however point you to some required reading on this topic, which may help you frame your idea: "Order your graphics calls around".

Tristam MacDonald. Ex-BigTech Software Engineer. Future farmer. [https://trist.am]

Quote:Original post by swiftcoder
I will however point you to some required reading on this topic, which may help you frame your idea: "Order your graphics calls around".


Thanks for the link, that seems like a really good method for storing sorting information and it's very flexible too.

Quote:I can't really provide a critique of your method, as you haven't really fleshed it out enough to be meaningful.


I guess I really should have asked a more specific question instead of throwing everything out there at once. The reason why that quick design is all I've got is because I'm not really sure where to go from here. I've got a renderable, it gets sorted using a render list and then the renderer receives the sorted renderables and draws them; that's all I've got.

It's the little things that irk me like how I can't give IRenderable methods like GetVertices() or GetNormals() for the renderer to use because the renderable might be something like a particle system or terrain. Also, where to apply transformations.

I've looked around, but I can't seem to find any good material on this sort of thing and having no experience in this, I'm pretty confused, so sorry about the really vague questions. =\
You could push the transformation onto the list as part of the renderable definition.

As for GetVertices and GetNormals, each renderable should have a VBO or similar.
Quote:Original post by -Datriot-
Quote:I can't really provide a critique of your method, as you haven't really fleshed it out enough to be meaningful.
I guess I really should have asked a more specific question instead of throwing everything out there at once. The reason why that quick design is all I've got is because I'm not really sure where to go from here. I've got a renderable, it gets sorted using a render list and then the renderer receives the sorted renderables and draws them; that's all I've got.

It's the little things that irk me like how I can't give IRenderable methods like GetVertices() or GetNormals() for the renderer to use because the renderable might be something like a particle system or terrain. Also, where to apply transformations.

I've looked around, but I can't seem to find any good material on this sort of thing and having no experience in this, I'm pretty confused, so sorry about the really vague questions. =\
I will try to fill you in a little on the structure of my renderer, as it may be of use. Keep in mind however that my renderer is most likely over-engineered, and is designed to solve a number of problems most people don't need to (i.e full-scale planets)...

We start with a scene graph. The scene graph is a pretty generic structure, and just allows nodes to be positioned hierarchically. Renderables may be attached to scene nodes, along with lights, cameras, etc.

Renderables (plus lights, cameras) also exist in a spatial structure, in this case a bounding interval hierarchy (a variant on kd-trees), which provides culling.

Every frame, I run an update on the scene graph, which updates the transformations of all child nodes. For each active camera (starting with the main scene camera), I run 'render request' over the spatial structure. This request performs culling, and may also spawn additional render requests, if another camera is found to be necessary (i.e. for render to texture).

During the culling process, each visible Renderable is asked to generate render commands, which are stored in a buffer attached to the camera. Each renderable may generate any number (including zero) render commands, and each render command contains a material (including textures, sorting, etc.), the necessary data for a render batch (i.e. vertex/index buffers and ranges), and some shader state (transformation matrices and other shader uniforms).

Finally, the cameras are ordered by dependency, and the render commands are submitted to the GPU in turn.




The most important point to take away from that (from your perspective) is probably that all render commands are structured identically, whether they are generated by the terrain renderer, the foliage renderer, or the GUI.

Also worth noting that the final stage which handles submitting commands to the GPU has no idea at all about the scene graph, spatial structure or render modules.

Tristam MacDonald. Ex-BigTech Software Engineer. Future farmer. [https://trist.am]

For what its worth, my graphics engine design:

A hierarchy of classes to represent a matrix transform, a matrix transform with attached geometry, a camera, a light, a group of objects. The base class (transform) contains methods for setting and getting its matrix both in local space and global space. It uses lazy evaluation in the global get function: If the matrix of a particular node is changed, its global transform and those of all its descendants are marked as requiring an update, which is then applied next time the get function is called.
I have a class called RenderStage that represents rendering to a single render target (or the screen) from a single camera. It contains a list of other RenderStages on which it depends so that the order of rendering can be determined. It can contain a list of inclusion meshes and a list of exclusion meshes the use of which will become clear here:
The main scene class is called SceneContext and it contains a list of all meshes in the scene as well as other info. During rendering the RenderStages are added to a list ensuring that dependency is adhered to, and then rendered in order. When a particular stage is rendered it can either render all meshes in the SceneContext, minus those in its exclusion list, or just those in its own inclusion list. In this way the list of objects rendered can be adjusted depending on needs. e.g. if rendering a reflection then maybe the RenderStage is set to only render the inclusion list and the land surface is added to it.

I've left out some other details like material sorting but those are handled how one would expect.

Quote:
As for GetVertices and GetNormals, each renderable should have a VBO or similar.


I found an old thread on this forum today that mentioned it's not good to have too much data in a single VBO. So, storing all the scene's geometry in a single VBO, like I initially thought, sounds like a bad idea. Each renderable using their own VBO seems to be the better option. It's also more manageable if each renderable handled its geometry. Instancing would also be easier since many renderables (that share the same mesh) could hold a reference to the same VBO.

Thanks for the tip. :)

Quote:
It uses lazy evaluation in the global get function: If the matrix of a particular node is changed, its global transform and those of all its descendants are marked as requiring an update, which is then applied next time the get function is called.


So a transform will only get updated when it actually needs to be used. I like that. :)

Quote:Every frame, I run an update on the scene graph, which updates the transformations of all child nodes.


If I understand correctly, each node's Update() method is called, which then updates all their children and all the transformations are updated by multiplying every child node's matrix with its parent.

If so, does this mean you update the matrices on the CPU? Wouldn't this be slow when a large amount of objects are added to the scene, or am I underestimating the power of modern CPUs?

Quote:
During the culling process, each visible Renderable is asked to generate render commands, which are stored in a buffer attached to the camera.


Ah, so if you have two or more cameras (for example, a different part of the scene is shown through a camera, rendered to a texture and drawn at the bottom right of the screen), the scene is culled twice, with render commands going to each camera. Pretty smart. I doubt I'll need more than one camera, but it's a useful to know nonetheless.

Quote:
the cameras are ordered by dependency


I'm not sure what you mean by 'ordered by dependency'.

Thanks to all of you for the help; I think I'm getting a better idea on how to structure this thing. I plan on slapping together a rough UML diagram tomorrow and I'll post it here so it can criticised.
Quote:Original post by -Datriot-
Quote:Every frame, I run an update on the scene graph, which updates the transformations of all child nodes.
If I understand correctly, each node's Update() method is called, which then updates all their children and all the transformations are updated by multiplying every child node's matrix with its parent.

If so, does this mean you update the matrices on the CPU? Wouldn't this be slow when a large amount of objects are added to the scene, or am I underestimating the power of modern CPUs?
My engine is designed to render entire galaxies, so the whole system works very hard to keep a minimal set of data in play at any one time.

However, the scene node update is fairly trivial to parallelise across multiple cores, and the update function is pretty lightweight - basically a single matrix multiply per node. There are also fairly few nodes in general, with one per entity (character, spaceship, etc.) and one for each LOD-system (terrain and vegetation systems manage sub-transforms internally).
Quote:
Quote:the cameras are ordered by dependency
I'm not sure what you mean by 'ordered by dependency'.
I just mean that if camera A can see a render-texture that is produced by camera B, then B must be rendered before A. In general this is fairly trivial, but there are some pathological cases (the classic 'hall of mirrors', for one).
Quote:Thanks to all of you for the help; I think I'm getting a better idea on how to structure this thing. I plan on slapping together a rough UML diagram tomorrow and I'll post it here so it can criticised.
Sounds good - always nice to see fresh approaches to these things.

Tristam MacDonald. Ex-BigTech Software Engineer. Future farmer. [https://trist.am]

Okay, I've created a rough draft of the design.

The SceneGraph does nothing more than store a root node and update all the nodes via Update(). SceneGraph::Update() calls the Update() method of the root node, passing in the identity matrix (since you have to pass some matrix in). The root node will then update its transformation and pass the updated matrix to all its children. Those children will then update their children, passing their new matrix.

I haven't decided on the details of the SpacialStructure yet, for now it's a structure that stores renderables and culls them. CullRenderables() takes the camera, decides what's visible and then returns a RenderCommandContainer (not the final name probably) that contains all of the renderable's RenderCommands. It is also responsible for sorting the renderables with the usual criteria (materials, depth, etc.).

The RenderCommandContainer's Sort() method is called and then the render commands are retrieved and sent to the Renderer, which sends the data to the GPU.

Oh, and Renderable, RenderCommandContainer, Renderer and SpacialStructure are interfaces so the culling/sorting/rendering routines can easily be changed. RenderCommand will also have more getters too and VBO/Texture/Shader are just some examples of the type of objects being used.

Of course, the design is missing a few nitty-gritty details, but I think it is good enough for an overview. Your thoughts?
How do you do those beautiful diagrams ?

This topic is closed to new replies.

Advertisement