GLSL skinning

Started by
11 comments, last by jmakitalo 13 years, 1 month ago
I'm trying to move the skinned vertex transformation from CPU to GPU. For rendering, I do following



glUniformMatrix4fvARB(shAttr->getSkinMatricesUniformLoc(), pDm->joints.size(), GL_FALSE, skinMatrices);

glBegin(GL_TRIANGLES);

for(int j=0; j<refGroup.faces.size(); j++){
CMD5Face &face = refGroup.faces[j];

for(int k=0; k<3; k++) {
CMD5VertexInst &vertex = group.vertices[face.vertexIndices[k]];
CMD5Vertex &refVertex = refGroup.vertices[face.vertexIndices[k]];


glVertexAttrib3fvARB(shAttr->getTangentAttribLoc(), refVertex.tangent.data());
glVertexAttrib4ivARB(shAttr->getBoneIndexAttribLoc(), refVertex.boneIndices);
glVertexAttrib4fvARB(shAttr->getWeightAttribLoc(), refVertex.weights);

glTexCoord2fv(refVertex.texCoords.data());
glNormal3fv(refVertex.normal.data());
glVertex3fv(refVertex.position.data());
}
}
glEnd();



The vertex positions are in bind pose and the skinMatrices is an array of floats with size maxBones*16*sizeof(float).
The skinMatrices consist of a product of the bone transformation to skinned pose and the inverse bind pose matrix.
I have tested that the matrices, weights and so on are ok by doing the transformation on CPU.

The vertex shader part that does the skinning is following:



uniform vec3 cameraPos;
uniform vec3 lightDir;
uniform int skinned;
uniform mat4 skinMatrices[50];

varying vec3 varDiffuse;
varying vec3 varAmbient;
varying vec3 varLightDir;
varying vec3 varHalfVector;

attribute vec3 tangent;
attribute ivec4 boneIndices;
attribute vec4 weights;

void main()
{

vec4 pos = vec4(0.0);
vec3 normal = vec3(0.0);
vec3 tangent2 = vec3(0.0);
if(skinned!=0){
for(int i=0; i<4; i++){
int index = boneIndices;
if(index>-1){
pos = pos + (skinMatrices[index]*gl_Vertex)*weights;
normal = normal + skinMatrices[index]*vec4(gl_Normal,0.0)*weights;
tangent2 = tangent2 + skinMatrices[index]*vec4(tangent,0.0)*weights;
}
}
}
else{
normal = normalize(gl_Normal);
tangent2 = normalize(tangent);
pos = gl_Vertex;
}


gl_Position = gl_ModelViewProjectionMatrix*pos;
}



I get nothing rendered. Can you see any flaws in the skinning related code?
Of course not all is shown above, but it would be nice to know if you find anything wrong with
what I have shown.
Advertisement
Try sending in the identity matrix for each bone matrix and see if you get the base model.

NBA2K, Madden, Maneater, Killing Floor, Sims http://www.pawlowskipinball.com/pinballeternal

This

glVertexAttrib4ivARB(shAttr->getBoneIndexAttribLoc(), refVertex.boneIndices);

needs to be :

glVertexAttribI4ivARB(shAttr->getBoneIndexAttribLoc(), refVertex.boneIndices);


All glVertexAttribN* functions convert the input value to a [-1,1] or [0,1] floating point value, so this won't work for index skinning.

The letters s,f, i,d, ub,us, and ui indicate
whether the arguments are of type short, float, int, double,
unsigned byte, unsigned short, or unsigned int. When
v is appended to the name, the commands can
take a pointer to an array of such values. The commands
containing N indicate that the arguments
will be passed as fixed-point values that are scaled to a
normalized range according to the component conversion rules
defined by the OpenGL specification. Signed values are
understood to represent fixed-point values in the range [-1,1],
and unsigned values are understood to represent fixed-point
values in the range [0,1].
[/quote]

Sending integers as vertex attributes was a feature added in SM 4.0, so you need the special extension functions from GL_EXT_gpu_shader4.
[size=2]My Projects:
[size=2]Portfolio Map for Android - Free Visual Portfolio Tracker
[size=2]Electron Flux for Android - Free Puzzle/Logic Game
Thanks for the suggestions.

I changed to glVertexAttribI4ivEXT and tried to pass identity matrices, but did not see the base mesh. This is puzzling.

How are the matrices supposed to ordered in a one-dimensional array of floats?
I assumed that column-by-column contiguously.
Maybe the way I tried to access the vector components was invalid, I changed this to



ivec4 index = boneIndices;
vec4 weight = weights;
for(int i=0; i<4; i++){
if(index.x<0) break;

pos = pos + (skinMatrices[int(index.x)]*gl_Vertex)*weight.x;
normal = normal + (skinMatrices[int(index.x)]*vec4(gl_Normal,0.0))*weight.x;
tangent2 = tangent2 + (skinMatrices[int(index.x)]*vec4(tangent,0.0))*weight.x;

index = index.yzwx;
weight = weight.yzwx;
}



But apparently this wasn't the only thing wrong, since it still wont work. Changing

pos = pos + (skinMatrices[int(index.x)]*gl_Vertex)*weight.x;

simply into

pos = pos + (vec4(1.0)*gl_Vertex)*weight.x;

did result with the base mesh, so I think this means that the weights must be correct, so
that they add up to one. The problem is either with the indices or the skin matrices.
I'm starting to feel like this could be a bug in the linux nvidia drivers. It just seems that the matrices
do not go through to the shader.
GL stores matrices as:

0 4 8 12
1 5 9 13
2 6 10 14
4 7 11 15

So your 1D array should be indexed properly.

NBA2K, Madden, Maneater, Killing Floor, Sims http://www.pawlowskipinball.com/pinballeternal

Can you try just drawing the values of the matrices to a color or something? It's quite a leap to assume that there must be a driver bug because your skinning doesn't work, it's generally a difficult thing to do with many many points of failure.

If you think the matrices are your point of failure, than you need to remove everything else from the equation to test it. Don't use indices, don't try to do transforms, don't rely on your vertex weights.

You need to be creative. How about just sending a matrix with all values = 0.77. Then you can just draw a quad with:

if(matrix[0][0] > 0.76 && matrix[0][0] < 0.78){
fragColor = vec4(1,0,0,1);
} else {
fragColor = vec4(0,1,0,1);
}

Use simple little tests to break apart your program till you find exactly what isn't what you expect.

Yes it's kind of annoying to do, but you can't just write a whole skinning shader, throw it all together, and expect it to work without any debug. There's many things that can go wrong.
[size=2]My Projects:
[size=2]Portfolio Map for Android - Free Visual Portfolio Tracker
[size=2]Electron Flux for Android - Free Puzzle/Logic Game
[font="Arial"]Ok, the problem was in my c++ code after all :D. A virtual function that was supposed to return attribute handle to bone indices, defined in a shader base class had a "const", which was missing in the actual implementation of the function. Now it works.

Now the challenge is to try getting this work with VBO:s. In opengl-doc.com it is said that when using int-type in glVertexAttribPointerARB,
the values are converted to float, and the user can decide whether the value is to be normalized. So should it be safe to use


glVertexAttribPointerARB(shAttr->getBoneIndexAttribLoc(), maxVertexWeights, GL_INT, 0, 0, 0);


for bone index arrays, and the indices in the vertex shader will be those that are actually stored in the VBO?
For some reason this is not working, but it might again be a problem in my c++ code.


So there is not equivalent to glVertexAttribI4iv for vertex arrays?


Thanks for all your replies.[/font]
I managed to get the VBO + GPU skinning to work in a counter-intuitive way:

by loading the bone index data as GLint:s with



glGenBuffersARB(1, &vboBoneIndex);
glBindBufferARB(GL_ARRAY_BUFFER_ARB, vboBoneIndex);
glBufferDataARB(GL_ARRAY_BUFFER_ARB, boneIndexArraySize*sizeof(GLint), boneIndexArray, GL_STATIC_DRAW);



and in the rendering code use GL_FLOAT for some bizarre reason, with



glEnableVertexAttribArrayARB(shAttr->getBoneIndexAttribLoc());
glBindBufferARB(GL_ARRAY_BUFFER_ARB, refGroup.vboBoneIndex);
glVertexAttribPointerARB(shAttr->getBoneIndexAttribLoc(), maxVertexWeights, GL_FLOAT, 0, 0, 0);



then it seems to work. If I use GL_INT in glVertexAttribPointerARB then it won't work. It will render one part of the mesh, though.
I think this happens, because the indices might be normalized or clamped for some reason, so that a bone with index zero will
be the only valid index.

If I use GLfloat also in glBufferDataARB, then it will not work again. I'm glad that I got this to work, but not that pleased because I don't understand
why it works.

This topic is closed to new replies.

Advertisement