Jump to content
  • Advertisement
john_connor

OpenGL how to systematicaly draw objects with different programs/VAOs/set of uniforms

Recommended Posts

hi,

i'm struggling with that question for a long time now. assume you have 2 programs (shader) and 2 vertex arrays to draw 2 different kind of things:

1. VAO: attributes are vec3 position, vec4 color

2. VAO: atributes are vec3 position, vec2 texturecoordinates, vec3 normals

both programs use different uniforms:

1. PROGRAM: mat4 transform

2. PROGRAM: mat4 transform, several material parameters, textures, lights, ...

drawcalls are the same for both ... how would you simplify that?

/// general draw call
struct DrawRangeElementsBaseVertex
{
	GLenum Mode = GL_POINTS;
	GLuint Start = 0;
	GLuint End = 0;
	GLsizei Count = 0;
	GLenum Type = GL_UNSIGNED_INT;
	const GLvoid* Indices = nullptr;
	GLint BaseVertex = 0;
};


/// general draw method
void Draw(const DrawElementsBaseVertex & cmd)
{
	glDrawElementsBaseVertex(cmd.Mode, cmd.Count, cmd.Type, cmd.Indices, cmd.BaseVertex);
}



/// (using PROGRAM 1 and VAO 1)
struct CDrawableNode
{
	/// world space transform
	glm::mat4 Transformation = glm::mat4(1);
	
	CMaterial Material;
	
	DrawRangeElementsBaseVertex MeshDrawCall;
};


/// (using PROGRAM 2 and VAO 2)
struct CDrawableGeometry
{
	/// world space transform
	glm::mat4 Transformation = glm::mat4(1);

	/// drawcall references
	std::vector<DrawRangeElementsBaseVertex> DrawCalls;
};

now i have 2 different for-loops for processing all the "CDrawableGeometry" and "CDrawableNode" in my scene. i'd like to do it with 1 big loop using "struct CRenderstates" somehow. has anybody some infos / links to useful sites / advices where to start with that ? somehow like:

for each renderstate ... set state

--> for each program/VAO ... use shader, bind program, set uniforms

----> for each object ... set program-specific uniforms, issue drawcalls

Share this post


Link to post
Share on other sites
Advertisement

Not sure what you're really asking, you pasted some source that doesn't really tell much - like what's your DrawableGeometry vs DrawableNode? What's the problem of doing it in one loop? Have you read about render queues, render items, sorting, state caching etc? 

Share this post


Link to post
Share on other sites
On 1/7/2019 at 12:41 PM, noizex said:

... what's your DrawableGeometry vs DrawableNode? What's the problem of doing it in one loop? Have you read about render queues, render items, sorting, state caching etc? 

lets say i want to draw 2 (or more) kinds of different object types:

first typ is just a usual textured mesh, using vertices with vec3 position / vec2 texcoord / vec3 normal, several of these objects / nodes are tied together in a single model (node tree / hierarchy). i want to have several different models (that means several different node trees).

second type is just a visual help, for example an x-z-grid to see the "ground level" y=0, and a coordinate axis to see the orientation of different nodes within a node tree (chained transformations). these use a vertex with vec3 position / vec4 color, no lighting applied, no fancy things added, just for help ...

of course that menas i have 2 different vertexarrays AND 2 different programs (or "shaders" if you want). for that, i have created 2 different kind (structs) of "drawable": DrawableNode and DrawableGeometry ... do you have any suggestions how to change that approach in a more systematic way ??

beside that, can you show me where to find further infos about "render queues", "render items" and so on ?!

... thanks in advance!

Share this post


Link to post
Share on other sites

First question: why you need two different types of structs to hold pretty similar data? Both cases require vertex array id, program id, primitive type, buffer ids etc. This means you could just have one "render item" struct that holds generic data about every mesh you want to draw. This allows further to store them in an efficient way and sort based on some criteria that would minimize the number of draw calls required. Remember - on this level it's all mesh drawn with some set of resources. 

If you need specific treatment for different meshes (like split drawing 2D/UI and regular 3D geometry, or have transparent geometry processed differently) you can create "lists" or "queues" which hold these uniform render items and can be processed on their own. 

For more read about these topics just use search on these forms for mentioned keywords ("render queue", "render items/atoms/tasks") and you will get plenty of very interesting topics where a lot of clever people described how they do this kind of stuff. Main and most known article about this, which is mostly focused on sorting these items is: http://realtimecollisiondetection.net/blog/?p=86

I think the main question you have to answer is what kind of granularity and how many levels of abstraction of render chunk you need - in initial post you mentioned Nodes but also DrawRangeElementsBaseVertex struct which seems like something that could be merged together and have it created by higher level rendering layer (the one that knows which meshes to draw per game object). So instead of having these two layers just combine them into one that has all the info - base per-mesh uniforms (matrices, material properties), textures, vertex/index buffers, primitive types etc. 

I can quickly describe what I have and what works fine for me, though by no means treat it as some kind of "best solution" - far from it.

I have 3 layers -

1. HIGH LEVEL INTERFACE which is directly accessed by my game logic, which contains container classes like StaticMesh, AnimatedMesh, BillboardMesh etc. These objects hold information such as Resource<Mesh> (which holds buffer ids etc.), Transform, visibility (drawn or not), and some specific logic like transforming mesh into another, having shape keys (morph targets) etc. There are two main roles for this layer:

a) It interfaces with game logic so I can affect the state of these objects from within my scripts, enabling, disabling, changing parameters as I need it. 

(scripting)
StaticMesh@ baseMesh("modelfile", transform);
baseMesh.effect = "outline";
baseMesh.enabled = false;

BillboardMesh@ tree("tree_sprite", transform);

b) From Renderer perspective, these objects belong to rendering system and are iterated each frame to construct lower level render tasks. Basically these objects know what to do to produce a 1..N render tasks that are required to "draw" them. 

(C++, render_system.cpp)
for (auto& mesh: staticMeshes_)
{
   mesh.render(renderQueue);
}

(C++, static_mesh.cpp)

void render(RenderQueue& queue)
{
   for (auto& mesh: model->getSubmeshes())
   {
      RenderTask task;
      task.vertexBuffer = mesh.vertexBufferId;
      task.indexBuffer = ...
      task.primitiveType = TRIANGLES;
      # and so on...
      queue.submit(SCENE_OPAQUE_GEOM, task);
   }
}

2. MID LEVEL - render queues and render tasks. So each frame renderer iterates existing objects (StaticMesh, AnimatedMesh, BillboardMesh) and collects RenderTasks from them. Each RenderTasks contains all the information needed to render some mesh and each object can create as many as it needs to "render itself". One important thing to note is that these tasks have nothing to do with draw calls YET. These tasks are then added to various queues - opaque scene geonetry, alpha tested geometry, skybox, 2D UI, shadow map etc - these lists are later processed by different "passes". 

3. LOW LEVEL (OpenGL calls) - once I construct queues and they're filled by render tasks, I begin various processing, and this heavily depends on passes that I have (which are another topic). Basically each pass may request one or more queues to "yield" their tasks (this does not result in clearing queue, so various passes can iterate over same sets of tasks) in a sorted way according to some sort key and algorithm. Based on this range of items, I perform batching which is done per-pass as different passes may batch differently. This means my sorted render tasks are processed one by one and I construct final draw call structs. This allows me to do things like - for 100 submitted render tasks, if they're all using same shader and buffers, I can stuff it all into a single indirect draw call. So the result of third layer is to produce draw calls - indirect calls, instanced calls, single calls - all based on criteria of sorting and batching.

Also on this level I not only have draw call commands but also state changes needed - setting current program or vertex buffers. So it's an ordered stream of commands which are then pushed onto GPU one after another. Worth noting is - it's not the top level interface that decides there should be SET_PROGRAM command, but the mid tier, and only after processing things through the batcher. This is needed because top level does not have information needed to properly decide on state changes and would cause a lot of unnecessary state changes, so I derive these commands only from the batcher which knows when program needs to change to another one. Basically state changes equal batch break in this case, so I have usually as many batches as state changes (buffers + programs + render states) combined.

Output of stage 3 is simple array of commands, that is then executed by GL context. It works pretty well so far and allows me to batch things like Sponza scene into just few indirect calls more or less automatically. I just submit all submeshes and let the system handle it. Let me know if you have any questions.

 

Share this post


Link to post
Share on other sites

sounds quite interesting, how complex is your "pipeline" ? do you have transformfeedbacks and other "non-trivial" routines ? however, it'd be interesting to take a look into your renderer :) ... after i thought about my question and your response a bit, i figured out the essence of my little problem:

different programs have different sets of "per-object-uniforms" (which is the reason for my 2 different structs that have about the same purpose). would you suggest to kind of use a "data-driven" approach (i hope thats the right word) meaning that i just have 1 struct with all possible per-object-uniforms for all programs? (redundand data)

regarding buffers, i dont mess with them anywhere else but in the renderer.cpp, since there is no reason to have as many buffers as objects. i have ALL meshes (of the same vertex type) in 1 buffer laid end to end. (of course thats only possible for static meshes)

Share this post


Link to post
Share on other sites
41 minutes ago, john_connor said:

sounds quite interesting, how complex is your "pipeline" ? do you have transformfeedbacks and other "non-trivial" routines ? however, it'd be interesting to take a look into your renderer :) ... after i thought about my question and your response a bit, i figured out the essence of my little problem:

different programs have different sets of "per-object-uniforms" (which is the reason for my 2 different structs that have about the same purpose). would you suggest to kind of use a "data-driven" approach (i hope thats the right word) meaning that i just have 1 struct with all possible per-object-uniforms for all programs? (redundand data)

regarding buffers, i dont mess with them anywhere else but in the renderer.cpp, since there is no reason to have as many buffers as objects. i have ALL meshes (of the same vertex type) in 1 buffer laid end to end. (of course thats only possible for static meshes)

Okay seems like new GD.net editor makes it not possible to easily split quotation into sections and reply below each one, so I will reply in a different way :/

re "sounds quite interesting, how complex is your "pipeline" ? do you have transformfeedbacks and other "non-trivial" routines"  - it's not very complex I'd say, but capable of any thing that I'd throw at it, and pretty flexible. I don't use TF right now, but I definitely will use compute shaders, and this will happen on "pass" stage - which is way too broad a topic to cover here. Basically all buffer handling happens either on resource load stage (vertex and index buffers for meshes) or on render/pass stage (filling uniform buffers and SSBOs with data each frame, or with material data whenever new material is loaded). Think of it this way: render queue provides all types of geometry to render, passes consume these items as they need (UI pass will consume 2d geom, alpha tested pass will render only alpha tested meshes etc.). If I need to put compute or transform feedback into it, it will happen on pass stage as well.

re "different programs have different sets of "per-object-uniforms" (which is the reason for my 2 different structs that have about the same purpose). would you suggest to kind of use a "data-driven" approach (i hope thats the right word) meaning that i just have 1 struct with all possible per-object-uniforms for all programs? (redundand data)"

That's something I wondered too. By going into UBOs/SSBOs you abandon easy way to freely define uniforms however you like, and you need to be more strict. Are you sure you need all these params, and if so, are they properties of materials? Or objects/meshes themselves? The way to have highly performant uniform buffers and indirect drawing later is to be pretty strict about these things, and it may pay off to rethink the approach. For example I ended up having very simple "per object" uniforms (matrices, material id, bone id if applicable) and these id types index into other uniform arrays (these can be smaller than max number of objects, mind it). So for things that control rendering I have material definitions with all needed properties etc. In the end you probably end up not needing that much unique properties per different "objects". 

re "egarding buffers, i dont mess with them anywhere else but in the renderer.cpp, since there is no reason to have as many buffers as objects. i have ALL meshes (of the same vertex type) in 1 buffer laid end to end. (of course thats only possible for static meshes)"

That's definitely a good thing to have. 

 

Share this post


Link to post
Share on other sites
15 hours ago, noizex said:

Okay seems like new GD.net editor makes it not possible to easily split quotation into sections and reply below each one, so I will reply in a different way :/

If you select part of the post, there should be a "Quote selection" button under it. You can do that multiple times and it will append them into the editor.

15 hours ago, noizex said:

That's definitely a good thing to have. 

For example :)

Or you could "Quote" the post multiple times and then edit the quotes, it should also append the quotes to the editor.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!