Hi, over the past week, I have been struggling alot with 3D vertex skinning. I use blender to create my model, to rig them and I expert the mesh and its armature in COLLADA file (.dae). Btw, I am using java with lwjgl, wich uses mostly opengl.

However, I am having trouble trying to get my mesh into its bind pose in my game engine.

Here is what I have :

In the <library_visual_scenes> node of the collada file, I can read the whole bone hierarchy, and get the local matrix of each joint:

<node id="Bone" name="Bone" sid="Bone" type="JOINT">
          <matrix sid="transform">1 0 0 0 0 7.54979e-8 -1 0 0 1 7.54979e-8 0 0 0 0 1</matrix>

Then, in the <library_controllers> node, I firstly read the bind shape matrix:

<skin source="#Cube-mesh">
        <bind_shape_matrix>1 -2.22045e-16 0 2.32831e-10 0 1 1.19249e-8 1.19249e-8 0 -1.19249e-8 0.9999999 0.9999999 0 0 0 1
</bind_shape_matrix>

In the same node, I also read all the inverse bind matrix aswell as all the weights associated to vertex.

I have verified all the datas after loading them to see if they get loaded correctly, so the problem should not be in loading datas, unless there is something I don't know about collada matrices.

After loading the collada file, I calculate the world matrices of each joint by multiplying its local matrix with its aprent's world matrix. For the root joint, the local matrix becomes the world matrix.

The vertex shader is where I calculate the final vertex position. Here is how I proceed (For some reason I cannot post code anymore) :

vec4 actualPosition = vec4(position, 1.0);
vec4 weightedPos = (((actualPosition * bindShapeMatrix) * boneTransforms[boneIDs.x]) * boneWeights.x) +
                                 (((actualPosition * bindShapeMatrix) * boneTransforms[boneIDs.y]) * boneWeights.y) +
                                 (((actualPosition * bindShapeMatrix) * boneTransforms[boneIDs.z]) * boneWeights.z) +
                                 (((actualPosition * bindShapeMatrix) * boneTransforms[boneIDs.w]) * boneWeights.w);
actualPosition += weightedPos;

Matrix4f skinningMatrix = new Matrix4f();
Matrix4f.mul(invBindShapeMatrix, worldMatrix, skinningMatrix);

As you can see, it is pretty close but something is definetly wrong.

For testing purpose, I have break it down to a simple cube with 2 bones. Bottom bone being the root bone, affecting the bottom 4 vertices of the cube, and the top bone affecting the other 4 top vertices. And I get the same results, some vertices (not all) are not at their position.

http://www.cjoint.com/15jv/EAqult0S5fE.htm

From this example, I definitely think something is wrong with the root bone, or something is wrong with the weights, because vertices affected by the top bones (4 top vetices) are in their correct position.

I am currently trying on paper to do all the matrix muliplications by myself to see if I get the same results. After doing the calculations in the shader, the vertex position should not change, so I am trying to see whats wrong. The down part of glsl is that I cannot put breakpoints or print stuff in the console to debug, so it makes things harder.

Sorry I hit the post button before finishing my post =/. Editing it right now ! :P

I would start by rendering some unskinned model just to make sure your data loading is correct.

Or just render the data your skinned geometry without the skinning transforms. You need to know if your vertices are in the world space or they have been saved after the bind pose has been removed (bone space/bind space?)

If the they are in world space, meaning if you draw them raw they look like the char in tpose or whatever, you usually will need to remove the bones bind pose first and then apply your animation transforms.

If they are not then drawing them raw they should look like a jumbled mess.

But my guess is that since you said you have the same bindShapeMatrix for every vert, the data could be in world space.

Otherwise you would need a bind pose transform for every joint to put them back in the correct spot. A single transform would just move the jumbled mess around.

For the parent joints make sure you are building the hierarchy from the root down, you can't just take a random joint and multiply by its parents world, if its parent world has not been transformed by its parent and so on back up to the root.

The same with your bones are they are in world space or local space related to their parent. Because their configuration will impact how your animations should be stored etc.

You will just have to work out the way your bones vs geometry is exported and adjust your code accordingly.

I would start with the simplest case draw it with no transforms. You could also write some code to draw your skeleton which can be useful to debug which aspect is broken.

And some configurations may be better than others in terms of performance etc.

But once its working in one configuration moving between them should be pretty straight forward.

I would start by rendering some unskinned model just to make sure your data loading is correct.

Or just render the data your skinned geometry without the skinning transforms. You need to know if your vertices are in the world space or they have been saved after the bind pose has been removed (bone space/bind space?)

If the they are in world space, meaning if you draw them raw they look like the char in tpose or whatever, you usually will need to remove the bones bind pose first and then apply your animation transforms.

If they are not then drawing them raw they should look like a jumbled mess.

But my guess is that since you said you have the same bindShapeMatrix for every vert, the data could be in world space.

Otherwise you would need a bind pose transform for every joint to put them back in the correct spot. A single transform would just move the jumbled mess around.

For the parent joints make sure you are building the hierarchy from the root down, you can't just take a random joint and multiply by its parents world, if its parent world has not been transformed by its parent and so on back up to the root.

The same with your bones are they are in world space or local space related to their parent. Because their configuration will impact how your animations should be stored etc.

You will just have to work out the way your bones vs geometry is exported and adjust your code accordingly.

I would start with the simplest case draw it with no transforms. You could also write some code to draw your skeleton which can be useful to debug which aspect is broken.

And some configurations may be better than others in terms of performance etc.

But once its working in one configuration moving between them should be pretty straight forward.

Thank you for the reply.

First of all, if I render my model without applying any skinning transforms, it looks exacly how it should look like.

I am not exacly sure but I am guessing that they are in world space, since they are draw a the right place.

In the file, the bones are relative to their parent bone. So the matrix I read is the local matrix, relative to its parent.

To create the world matrix or a joint I take the parent world matrix and I multiply it by its local transform. I already start from the root bone, and move down the hierarchy. As for the root bone, its local transform becomes the world transform.

I have thought about trying to draw my skeleton to see if there is a problem, but I am still not an expert with openGL drawing stuff, but I will work on that.

Other than that, I have just no idea what else to try. I have switch matrices around to see the result, but nothing is as close as what I currently get.

You basically have two options here

1. Your verts are in worldspace that means you need to transform them first by the original/binding bones pose.

So what you end up with is your verts in bone space.

You would construct your bind skeleton in world space and then invert it.

Then transform all your verts with the inverted version of their bones using the correct weightings.

This will put them into the space of the bone space.

Then you can construct your new skeleton from animations etc in world space. Then transform the bone space verts by that skeleton and they should move back around your new skeleton.

So you basically subtracting out the bind skeleton then adding back in the animated skeleton.

If you do this correct you should be able to apply the bind skeleton back and it should leave the verts unchanged.

2. If you don't want to do this to your verts you can multiply the inverse of the bind skeleton to your animated skeleton to get the change in pose per bone and then apply that to your verts still in world space.

I usually do #1 so I can avoid the extra multiplication of the bind skeleton every time, since you only need to subtract out the bind skeleton once.

You confused me a bit there. I am not actually sure what the bone space and world space.

The vertices I load from the collada file are in local space (I think).

Does that mean I can apply the inverse of the bone transform direcly to the vertices to put them into bone space. From there, I apply the new bone transform (im assuming there is only one bone here.) and ti would put them back into 'local space'.

Also, do you know what the bind shape matrix is for ? I read everywhere that you have to multiply the vertex with this bind shape matrix, and I am unsure what it is exacly for

EDIT

Now I think I understand a bit better. You need to apply the inverse of the bind pose in order to pring the vertex back to "0" then aply the current transformation matrix. Am I right ?

I can read an inverse bind matrix from the COLLADA, but shouldnt it be the same as if I take the absolute transform matrix of a joint in its bind pose and inverse it ? Because it isnt right now. I currently have only one bone in my model, and here is the data I read from the COLLADA file :

bone transformation matrix :

I got it working with a single bone affecting all vertices of a cube by 1. (!!)

I followed the whole process again starting from reading matrices in the collada file, and I realised there was something inversed in the translation part of the matrices. When building my matrices from a string (EX : 1 0 0 0 0 -1 5.88609e-8 0 0 -5.88609e-8 -1 0 0 0 0 1), the Y and Z translations were switched together. Thank you very much Eklipse your reply made me understand a lot of things that I thought I understood before. It is now much clearer how the matrices work together.

Cool. Good to hear.

Understanding your data is very important. Otherwise you are just guessing what is going on. And build simple test cases in your modeling package and make sure they work.

So if you have it working with 1 vert then try 2 next.

Bone space is just the space after you remove the bones transform.

Basically the vertex positions are relative to the bone(s) they are attached too. Then you can just transform them by a new animated etc bone transform to put them back into world space or whatever space your skeleton is in.

I also almost mentioned that to make sure your transforms ordering is correct from the components.

Also make sure your matrices are multiplied in the correct order parent * child or child * parent when building your skeleton. Start with simple transforms so you can inspect it make sure the values look correct.