Hey everyone, I'm trying to implement my shader system and I was previously hardcoding uniforms but I want to go for a more dynamic approach. I found out about Uniform Buffers. So as I understand, what you want is to potentially have a uniform buffer for the Camera and its properties, a uniform buffer for a potential material, etc. Here is the issue I'm running into.

I have a shader program class and in the shader program class, I want it to store a list of all uniform buffer attributes inside the actual program. So basically, each uniform buffer attribute would tell you the number of indices, the buffer name, the buffer byte size, the offsets, the names of the indices, etc. The problem here is that the buffer index in the program isn't predetermined (if it can be, it still would cause issues since you would have to make sure index 1 is the same for all shader programs). So I wanted to get an output array after linking that would state all the uniform buffer names in the program. From there, I could query all the details I need for each uniform buffer and put that into a Uniform Buffer Attribute class.

When rendering, the Renderable object would have its list of uniform buffers and would check to see if the properties of each uniform buffer in the renderable match that which is in the shader program that it is using to render itself. If true, it would bind it over and if not, I would get an warning.

Unfortunatly, I couldn't find a way online to query the names of all uniform buffers in a shader program. Instead, everyone puts in constant strings since they know before hand what uniform buffers are in the shaders. This doesn't work well since I want my c++ program to easily switch between shader programs/ uniform buffers for each renderable.

An issue that I see with my approach as well is that the same uniform buffer can have different binding indexes for different programs which could remove the benefit of not having to rebind the uniform buffer after switching shader programs.

I'm not sure if there is some general way around this but I couldn't find a answer online. I was looking a lot at this tutorial (http://www.gamedev.net/page/resources/_/technical/opengl/opengl-40-using-uniform-blocks-and-uniform-buffer-objects-r2860) and other links on different sites but they all use glGetUniformBlockIndex with a constant uniform buffer name. Here's how my code looks for my uniform buffer attribute:

class GLUniformBufferAttributes
	{
	private:
		GLuint m_BufferSize;
		std::string m_BufferName;

		std::vector <GLuint> m_UniformOffsetArray;
		std::vector <std::string> m_UniformNameArray;

	public:
		GLUniformBufferAttributes ();
		~GLUniformBufferAttributes ();

		bool SetAttributes (GLuint bufferSize, std::string & bufferName, const std::vector<GLuint> & offsetArray, 
							const std::vector<std::string> & nameArray);
	};

My uniform buffer class is sparse and probably has errors since I haven't tried populating it yet but instead getting the querying of attributes to work properly first. As of now, I'm following this link (http://gamedev.stackexchange.com/questions/48926/opengl-fetching-the-names-of-all-uniform-blocks-in-your-program) which does something similar with the index's. The query buffer name should probably be done before the loop but I couldn't find a function to do it. Here's how I'm trying to query the data in my shader program class after linknig:

class GLShaderProgram
	{
	private:
		GLuint m_ShaderProgId;
		std::vector <GLUniformBufferAttributes> m_UniformAttributeArray;
        };

bool GLShaderProgram::InitializeUniformBlocks ()
	{
		// Get all uniform buffer specifications
		GLint numUniformBlocks; // Get number of uniform buffers
		glGetProgramiv(m_ShaderProgId, GL_ACTIVE_UNIFORM_BLOCKS, &numUniformBlocks);

		m_UniformAttributeArray = std::vector <GLUniformBufferAttributes> (numUniformBlocks, GLUniformBufferAttributes());
		
		for ( unsigned int index = 0; index < numUniformBlocks; index++ ) // unsafe unsigned vs signed int mistmatch
		{
			// Query buffer size
			GLint blockSize;
			glGetActiveUniformBlockiv(m_ShaderProgId, index, GL_UNIFORM_BLOCK_DATA_SIZE, &blockSize);

			// Query buffer name
			GLchar bufferName;
			

			// Query number of uniforms
			GLint numUniforms;
			glGetActiveUniformBlockiv(m_ShaderProgId, index, GL_UNIFORM_BLOCK_NAME_LENGTH, &numUniforms);

			// Query the names of uniforms
			std::vector <GLchar> nameArray;
			nameArray.resize(numUniforms);
			glGetActiveUniformBlockName(m_ShaderProgId, index, numUniforms, nullptr, &nameArray[0]);

			// Query the offsets of uniforms
			std::vector <GLint> offsetArray;
			glGetActiveUniformBlockiv(m_ShaderProgId, index, GL_UNIFORM_OFFSET, &offsetArray[0]);

			//m_UniformAttributeArray[index].SetAttributes(bufferSize, bufferName, offsetArray, nameArray);
		}

		return true;
	}

#version 330

layout (location = 0) in vec3 Position;
layout (location = 1) in vec3 Normal;
layout (location = 2) in vec2 TexCoords;

uniform TestBlock
{
	vec4 v1;
	float r1;
};

uniform mat4 gWorld;

out vec2 TexCoord;

void main()
{
    gl_Position = gWorld * vec4(Position, 1.0);
	TexCoord = TexCoords;
}

Another note, I have a uniform defined in my vertex shader which itself executes properly, however I'm assuming the index loop is messed up since when it gets the uniform buffer at index 0, it pulls out a completely different uniform buffer which I'm assuming is the default one in shader programs since it has weird variable names and what not. It has 10 variables in it and they're names are single letters + numbers. Thanks in advance.

I'm dealing with this issue by suiting the graphics rendering pipeline and the shader scripts together. A specific kind of render pipeline has its specific stages: Forward lighting, material rendering, screen-space lighting, tone mapping, image post-processing, ... whatever. There are several kinds of properties needed: Material properties for physical based shading (or the suitable NPR properties for NPR rendering; let's call this material, too), camera set-up, screen rect, lights, and so on. Different stages have (partially) distinct requirements, but inside a stage the requirements are fixed.

So here is the deal: A camera provides a uniform block with its parameters, the screen provides one with its own parameters, the model provides one with its material, a post-process provides one with its parameters, a pipeline stage provides one, and perhaps other do so. Each block coming from such a source has an equivalent in the respective shader scripts, and there is a standard name used for it: E.g. CameraSetup, ScreenRect, Material, ... So it is convention that every script implementing forward rendering or material rendering has a slot for e.g. a Material block. Hence one can expect to have blocks with a specific name in a script suitable for a specific kind of rendering stage.

The higher level of rendering is implemented inside the stages of the render pipeline. It fetches the suitable blocks from the various sources and generates suitable binding instructions in the state part of the rendering jobs. The lower level of rendering (i.e. the one that calls OpenGL) then has to process those without the need to know the specific meaning of the parameters in a block.

Perhaps it is worth to mention that I use std140 layout. This makes things easier at least if the block is shared over many calls, like e.g. CameraSetup.

I do it in the following way:

My UniformBuffer has a usage flag, which is "Default" or "Shared". Now we have 2 scenarios on using the UBO:

1) If the UniformBuffer usage is "Shared", then the UBO send an event to the ShaderManager to register this UBO to all the loaded shaders. In my engine it's guaranteed that all the shaders are pre-loaded, so when you create a shared UBO, it's also guaranteed that all the shaders will know about it. When the renderer is binding a shader, it also binds its registered UBOs.

2) If the UniformBuffer usage is "Default", then I manually register the UBO to whichever shader I want.

I use the std140 layout as well.

I am not sure if this approach is 100% correct though. Reading haegarr's approach makes me re-think of my design a bit.

I'm dealing with this issue by suiting the graphics rendering pipeline and the shader scripts together. A specific kind of render pipeline has its specific stages: Forward lighting, material rendering, screen-space lighting, tone mapping, image post-processing, ... whatever. There are several kinds of properties needed: Material properties for physical based shading (or the suitable NPR properties for NPR rendering; let's call this material, too), camera set-up, screen rect, lights, and so on. Different stages have (partially) distinct requirements, but inside a stage the requirements are fixed.

So here is the deal: A camera provides a uniform block with its parameters, the screen provides one with its own parameters, the model provides one with its material, a post-process provides one with its parameters, a pipeline stage provides one, and perhaps other do so. Each block coming from such a source has an equivalent in the respective shader scripts, and there is a standard name used for it: E.g. CameraSetup, ScreenRect, Material, ... So it is convention that every script implementing forward rendering or material rendering has a slot for e.g. a Material block. Hence one can expect to have blocks with a specific name in a script suitable for a specific kind of rendering stage.

The higher level of rendering is implemented inside the stages of the render pipeline. It fetches the suitable blocks from the various sources and generates suitable binding instructions in the state part of the rendering jobs. The lower level of rendering (i.e. the one that calls OpenGL) then has to process those without the need to know the specific meaning of the parameters in a block.

Perhaps it is worth to mention that I use std140 layout. This makes things easier at least if the block is shared over many calls, like e.g. CameraSetup.

I do it in the following way:

My UniformBuffer has a usage flag, which is "Default" or "Shared". Now we have 2 scenarios on using the UBO:

1) If the UniformBuffer usage is "Shared", then the UBO send an event to the ShaderManager to register this UBO to all the loaded shaders. In my engine it's guaranteed that all the shaders are pre-loaded, so when you create a shared UBO, it's also guaranteed that all the shaders will know about it. When the renderer is binding a shader, it also binds its registered UBOs.

2) If the UniformBuffer usage is "Default", then I manually register the UBO to whichever shader I want.

I use the std140 layout as well.

I am not sure if this approach is 100% correct though. Reading haegarr's approach makes me re-think of my design a bit.

I see, it still doesn't make sense how you would know if the uniform buffers provided fulfill the shader program requirments.

Okay so basically you avoid the whole problem of having to get a uniform buffer's index by making sure all shaders follow a naming convention for their uniform buffers. But what happens when I have a model which in the game, had something happen to it where before it was being rendered by a certain shader program but now were making it rendered by another shader program. This new shader program takes in 4 uniform buffers while the previous one took in 3. Do you just assume that while the renderable was being switched from one shader program to another, that whatever switched it knew it had to provide a new uniform buffer?

A shader script is part of a rendering pipeline. As such a specific script has to do what the pipeline needs at the stage where the script is used, and the script can use whatever the stage (directly and indirectly) provides but nothing more. Moreover, there is a correspondence of the scripts functionality and the parameters provided by the uniform buffers. For example, if a game entity should be rendered by using the Gooch NPR technique, the entity must provide the suitable parameters in its "material" block. Switching over to the Toon shader requires the material block to be switched, too.

In fact I don't try to avoid requesting indexes of uniform blocks (which can simply be requested once and stored for later use). Instead I define how scripts have to work in specific situations. So I know which names to use for the uniform blocks of scripts of a specific type. If a particular script can be replaced by another one where the latter uses an additional parameter block, then it means that the former script simply does without it because it doesn't need it although it could use it. In other words, I don't allow for arbitrary scripts because they would not fit into the bigger architecture.

I see, it still doesn't make sense how you would know if the uniform buffers provided fulfill the shader program requirments.

Personally, if the shader does not contain the specific uniform buffer name (using glGetUniformBlockIndex), I ignore it/send a warning to the console.

Moreover, my renderer is a deferred renderer so I more or less know exactly what kind of shaders I have at any given time.

Okay so basically you avoid the whole problem of having to get a uniform buffer's index by making sure all shaders follow a naming convention for their uniform buffers. But what happens when I have a model which in the game, had something happen to it where before it was being rendered by a certain shader program but now were making it rendered by another shader program. This new shader program takes in 4 uniform buffers while the previous one took in 3. Do you just assume that while the renderable was being switched from one shader program to another, that whatever switched it knew it had to provide a new uniform buffer?

A shader script is part of a rendering pipeline. As such a specific script has to do what the pipeline needs at the stage where the script is used, and the script can use whatever the stage (directly and indirectly) provides but nothing more. Moreover, there is a correspondence of the scripts functionality and the parameters provided by the uniform buffers. For example, if a game entity should be rendered by using the Gooch NPR technique, the entity must provide the suitable parameters in its "material" block. Switching over to the Toon shader requires the material block to be switched, too.

In fact I don't try to avoid requesting indexes of uniform blocks (which can simply be requested once and stored for later use). Instead I define how scripts have to work in specific situations. So I know which names to use for the uniform blocks of scripts of a specific type. If a particular script can be replaced by another one where the latter uses an additional parameter block, then it means that the former script simply does without it because it doesn't need it although it could use it. In other words, I don't allow for arbitrary scripts because they would not fit into the bigger architecture.

Okay I'm slightly understand this. I haven't seen a shader script before but from what I understand, you get the parameters for each uniform buffer from the shader script which also describes how the uniform buffers must be set when you switch from one shader script to another. But then you need to specify how a shader script must populate its uniform buffers from all possible switches between shader scripts. So switching between toon/phong/physically based shading requires each shader script for toon/phong/... to have info on how to set its uniform buffers when switching between any of the other types of shaders. Doesn't that kind of reliance on other shader scripts cause a lot of potential for errors or am I completely misunderstanding this?

I see, it still doesn't make sense how you would know if the uniform buffers provided fulfill the shader program requirments.

Personally, if the shader does not contain the specific uniform buffer name (using glGetUniformBlockIndex), I ignore it/send a warning to the console.

Moreover, my renderer is a deferred renderer so I more or less know exactly what kind of shaders I have at any given time.

Sorry I should have rephrased it, what happens if you don't give a required uniform buffer for a shader because the object being rendered doesn't have it? In that case, you can't figure out if one of the uniform buffers needed for the shader hasn't been given since you can't query the names of all uniform buffers in a shader. Its weird that I can't query that since I can do that for uniforms, is there maybe a variant of glGetProgramiv that lets me get all the uniform buffer names in a shader program?

Okay I'm slightly understand this. I haven't seen a shader script before but from what I understand, you get the parameters for each uniform buffer from the shader script which also describes how the uniform buffers must be set when you switch from one shader script to another. But then you need to specify how a shader script must populate its uniform buffers from all possible switches between shader scripts. So switching between toon/phong/physically based shading requires each shader script for toon/phong/... to have info on how to set its uniform buffers when switching between any of the other types of shaders. Doesn't that kind of reliance on other shader scripts cause a lot of potential for errors or am I completely misunderstanding this?

Let us assume that the toon shading script, Phong shading script and the physically based shading script are all implemented as forward lighting scripts. The set of parameter used by a toon shader differ from the set of parameters used by a Phong shader and differ from the set of parameters used by a physically based shader. However, all those shaders have a block which can be understood as a material which encapsulates such set of parameters although the sets themselves differ in their structure and content.

The graphic rendering pipeline implements forward lighting. Hence each of the said shader scripts fit into one of the stages of the pipeline. This stage of the pipeline is aware that scripts plugged into it may be interested in something that is a material uniform block (besides others like a CameraSetup, Ligths, etc; but let us stay with the material, because it is the only one that may change due to what kind of shading the script implements).

When it comes to rendering, the particular pipeline stage is fed with the list of models (which have passed the visibility culling and are already sorted for minimal state switching costs; but that is another topic). The stage "asks" the next model which shading to be used and it requests the material block to be used. Notice that this step defines the script and the uniform block. The pipeline stage does not investigate the material block. Instead it looks at it as a black box. It simply binds it to the shader script as the block named "material". In other words, it relies on the kind of script, the script itself, and the provided material block fit together.

Obviously, if the kind of shading is switched, the actual material parameter block has to be switched, too. Details of the process are implementation dependent, of course. For example, you may have a constant field in the C/C++/Java/... implementation of the material that directly or indirectly refers to the kind of shading. Then switching to another kind of material automatically refers to another kind of shading and hence scripts.

Hey, sorry for such a late reply. Midterms came along and ate way more time than I thought they would.

So using the shader script, you assume that the given parameters, block names, and what not match up with the required uniform buffers for a shader program. It also seems like your shader program stores a list of buffers that it requests from the Renderable, and then binds them to the proper indices (which I’m probably going to do too, seems to work really well). Okay so I did some wrapping and implementation and got my GLShaderProgram class to support my GLUniformBuffer class. As shown before, my Uniform buffer class stores a class called UniformBufferAttributes which holds the name of the Uniform Buffer, the size of the uniform buffer, and the array of offsets. Originally, I also wanted GLShaderProgram to store a list of UniformBufferAttributes which it would then use to compare each uniform buffer thats being binded. That way, I could verify before binding whether or not I'm passing the properly encoded uniform buffer to the shader instead of passing and then waiting for an error.

I still think this process has some benefits since the drivers might let the Uniform Buffer shader run even if it’s missing some of the parameters. As an example, if I give 3 vertex locations in my shader but only have 2 in the actual data + only 2 enabled using glEnableVertexAttribute, the program still renders the cube correctly on my hardware. I'm assuming other driver implementations might throw an error so I'm debating keeping Uniform Buffer attributes just for proper error checking.

On the opposite side of the argument, storing the offsets to do comparisons each frame for each UBO is probably a lot of extra overhead just for error checking, so instead of having uniform buffer attributes, I might be better off with only keeping the UBO name and letting OpenGL throw the error itself. I'm not sure if it will consistently come up on glGetError if not enough parameters are given to the UBO or if it will just give 0 values and run.

As for the shader scripting part to give parameters to UBO's, I think I might go with having some kind of special Uniform Buffers. My idea now is that via shader script (which I'm assuming is just some XML file stored with a model we are loading), I can give init values for all my required parameters, however certain UBO's will change every frame. An example is a UBO for orientation, it needs to have the orientation updated every frame so we need a way of distinguishing it from other UBO's.

Lastly, about the example of switching lighting from Phong to Physically Based to Toon, I think in my implementation, it would make sense if the shader script had a uniform buffer specified for each case. So then when the shader requests a uniform buffer, it just pulls the Toon shader one vs the others since the names could be different. Then as a backup, there could be a default uniform buffer created by the GLShaderProgram if no toon shader buffer is provided.

I think I'm starting to get how the whole thing should work :D