I am working on a renderer that can manage drawing a lot of instances of 3d hexagon tiles that can have a different material on each face (well just about each face anyway)

in rendering instances of each tile I use a VBO that contains 4 by 4 matrix transforms which are made up of four 4 dimensional vectors - each vector being a row of the transform matrix, and then I assign shader attributes to each vector and use glVertexAttribDivisor to make the transforms per instance rather than per vertex - so that it kind of looks like this

        glBindBuffer(GL_ARRAY_BUFFER, transformBuffer);

	glEnableVertexAttribArray(TRANS1_ATT);
	glVertexAttribPointer(TRANS1_ATT, 4, GL_FLOAT, GL_FALSE, sizeof(NSMatrix4Df), 0);
	glVertexAttribDivisor(TRANS1_ATT, 1);

	glEnableVertexAttribArray(TRANS2_ATT);
	glVertexAttribPointer(TRANS2_ATT, 4, GL_FLOAT, GL_FALSE, sizeof(NSMatrix4Df), (const GLvoid*)(sizeof(NSVec4Df)));
	glVertexAttribDivisor(TRANS2_ATT, 1);

	glEnableVertexAttribArray(TRANS3_ATT);
	glVertexAttribPointer(TRANS3_ATT, 4, GL_FLOAT, GL_FALSE, sizeof(NSMatrix4Df), (const GLvoid*)(sizeof(NSVec4Df) * 2));
	glVertexAttribDivisor(TRANS3_ATT, 1);

	glEnableVertexAttribArray(TRANS4_ATT);
	glVertexAttribPointer(TRANS4_ATT, 4, GL_FLOAT, GL_FALSE, sizeof(NSMatrix4Df), (const GLvoid*)(sizeof(NSVec4Df) * 3));
	glVertexAttribDivisor(TRANS4_ATT, 1);

I just use the attribute divisor call to tell opengl to advance through the buffer once per instance rather than once per vertex (as each instance needs a transform not each vertex)..

So my question is this... I can potentially have a very large number of instances of one submesh - worst case scenerio 64 by 64 by 64 = 262,144 instances of a single submesh (really would never happen none the less that is the maximum).. Because I have a toolkit that lets the user place tiles in the map in real time the transformBuffer can potentially change once per frame

When creating and filling this transformBuffer, what do you think would be the best way to minimize runtime overhead? Make the buffer a certain size and use glBufferSubData and then resize the buffer only when the number of instances gets above the buffer size (kind of like how a c++ std::vector works)?

You mean you can have 262K model matrices? Thats still just 16 MB ....you can make a VBO with that size without having to worry.

The std::vector kind of solution is more elegant, but as probably not many people are going to manually place 100thousand or so objects, ....you can just set a limit for the maximum number objects and use a fixed size VBO.

On the other hand, if this is an editor, not an actual game, ....it is more open for some kinds of optimizations, and some are not relevant.

yeah 262 k model matrices - and you are correct about not placing that many - the thing is that I have a tool for placing 64 by 64 at one time (so that they can create an entire ground layer at once) and the user could potentially place 64 of these on top of eachother since I allow for map sizes of up to 64 by 64 by 64 hex tiles..

Before I was using a separate buffer for each hex tile mesh - that would mean if the user created say, 20 different hex tile meshes then that would be about 320 mb (if each VBO is 16 mb)

but that is a good idea to use a single buffer for all the transforms and just find which hex tile im rendering in the shader - that way I could just use a single VBO of about 16 mb and that would hold enough model matrices no matter how many hex tile meshes the user would decide to make

thanks!

I cant really imagine what this looks like, so Im juts guessing.

If its something like a map built up from voxels, so they cant rotate freely around all axes, then you dont need all that transformation data. In any case the last column or row of the matrix is just 0,0,0,1 so you dont have to send that to the shader.

Also, if they are just plain hexagon tiles made of 6 triangles, you could just apply the transformation to the vertices and put them into the VBO like that.

It sounds like a 3 element translation vector would be enough as all tiles would probably have same size and same rotation, no need for a 4x4 matrix then.