Sign in to follow this  
yanjobin2641

3D vertex skinning

Recommended Posts

Hi, over the past week, I have been struggling alot with 3D vertex skinning. I use blender to create my model, to rig them and I expert the mesh and its armature in COLLADA file (.dae). Btw, I am using java with lwjgl, wich uses mostly opengl.

 

However, I am having trouble trying to get my mesh into its bind pose in my game engine.

Here is what I have :

 

In the <library_visual_scenes> node of the collada file, I can read the whole bone hierarchy, and get the local matrix of each joint:

<node id="Bone" name="Bone" sid="Bone" type="JOINT">
          <matrix sid="transform">1 0 0 0 0 7.54979e-8 -1 0 0 1 7.54979e-8 0 0 0 0 1</matrix>

Then, in the <library_controllers> node, I firstly read the bind shape matrix:

<skin source="#Cube-mesh">
        <bind_shape_matrix>1 -2.22045e-16 0 2.32831e-10 0 1 1.19249e-8 1.19249e-8 0 -1.19249e-8 0.9999999 0.9999999 0 0 0 1
</bind_shape_matrix>

In the same node, I also read all the inverse bind matrix aswell as all the weights associated to vertex.

 

I have verified all the datas after loading them to see if they get loaded correctly, so the problem should not be in loading datas, unless there is something I don't know about collada matrices.

 

After loading the collada file, I calculate the world matrices of each joint by multiplying its local matrix with its aprent's world matrix. For the root joint, the local matrix becomes the world matrix.

 

The vertex shader is where I calculate the final vertex position. Here is how I proceed (For some reason I cannot post code anymore) :

vec4 actualPosition = vec4(position, 1.0);
vec4 weightedPos = (((actualPosition * bindShapeMatrix) * boneTransforms[boneIDs.x]) * boneWeights.x) +
                                 (((actualPosition * bindShapeMatrix) * boneTransforms[boneIDs.y]) * boneWeights.y) +
                                 (((actualPosition * bindShapeMatrix) * boneTransforms[boneIDs.z]) * boneWeights.z) +
                                 (((actualPosition * bindShapeMatrix) * boneTransforms[boneIDs.w]) * boneWeights.w);
actualPosition += weightedPos;

 

The bindShapeMatrix was taken from the collada file. It's the same for every vertex.
The boneIDs are indices to get the skinningMatrix from boneTransfroms
The boneWeights are weights that the corresponding boneID affect the current vetex. (total of 1.0, for all 4)
The boneTransforms array contains all the joints skinning matrices(I think thats how they call it)
 
To get the skinning matrix of a joint, I multiply its inverse bind matric with its world matrix :
 
Matrix4f skinningMatrix = new Matrix4f();
Matrix4f.mul(invBindShapeMatrix, worldMatrix, skinningMatrix);
 
But sadly, this doesnt give me the right results. On the left you can see an image of what it should look like in its bind pose, taken from blender. On the right, there is 2 images of different views of what it actually look like in my game engine, 
 

 

As you can see, it is pretty close but something is definetly wrong.

 

For testing purpose, I have break it down to a simple cube with 2 bones. Bottom bone being the root bone, affecting the bottom 4 vertices of the cube, and the top bone affecting the other 4 top vertices. And I get the same results, some vertices (not all) are not at their position.

 

http://www.cjoint.com/15jv/EAqult0S5fE.htm

 

From this example, I definitely think something is wrong with the root bone, or something is wrong with the weights, because vertices affected by the top bones (4 top vetices) are in their correct position.

 

I am currently trying on paper to do all the matrix muliplications by myself to see if I get the same results. After doing the calculations in the shader, the vertex position should not change, so I am trying to see whats wrong. The down part of glsl is that I cannot put breakpoints or print stuff in the console to debug, so it makes things harder.

Edited by yanjobin2641

Share this post


Link to post
Share on other sites

Hi, over the past week, I have been struggling alot with 3D vertex skinning. I use blender to create my model, to rig them and I expert the mesh and its armature in COLLADA


Acknowledged.
 
 
L. Spiro
 
 
[EDIT]
He added more to his post.
[/EDIT] Edited by L. Spiro

Share this post


Link to post
Share on other sites


I think I might have hit a button that hide this toolbar.

 

In the upper-right hand corner of the editor is a small arrow button which hides/displays the toolbar.

 

In any case, a picture will likely be no help. 3D vertex skinning is, perhaps, one of the most difficult intermediate-to-advanced processes to understand and debug. A picture of some craziness is unlikely to indicate what in your code or in your data (perhaps both) is incorrect.

 

So, some tough love regarding your post (we've all been where you are wink.png ).

 

First, read the Beginner's Forum FAQ. In particular, note the guidance under "Asking Questions" - be prepared to answer the question: what have you done to determine the problem yourself? Consider that asked.

 

Second, there are two necessary parts to any program - the code, and the data the code manipulates. Both must be correct to get the correct results. If you're not getting the expected results, you, as the programmer, should determine at what point in your code the expected results are incorrect. One method is to follow the data through the code during execution of the program. You may be lucky and someone will post a response here in the next few hours or days that will "fix" your problem. However, now and in the future, learning how to debug will save you hours (if not days).

 

Third, you should be debugging using the simplest test case (e.g., a model with just one or two bones, just a handful of vertices, with a simple animation - i.e., no movement).

 

You've posted some data, and posted some code. Help others help you by being able to state: "I've verified the data gets loaded correctly by examining the actual values. I've verified the code manipulates that data correctly up to this point in my code. With the data shown here, these [three-or-four] lines of code result in this incorrect data." Then you should ask: "How do I find out why the code does that?" rather than "What's wrong?"

Share this post


Link to post
Share on other sites

I would start by rendering some unskinned model just to make sure your data loading is correct.

 

Or just render the data your skinned geometry without the skinning transforms. You need to know if your vertices are in the world space or they have been saved after the bind pose has been removed (bone space/bind space?)

 

If the they are in world space, meaning if you draw them raw they look like the char in tpose or whatever, you usually will need to remove the bones bind pose first and then apply your animation transforms.

 

If they are not then drawing them raw they should look like a jumbled mess.

 

But my guess is that since you said you have the same bindShapeMatrix for every vert, the data could be in world space.

Otherwise you would need a bind pose transform for every joint to put them back in the correct spot. A single transform would just move the jumbled mess around.

 

For the parent joints make sure you are building the hierarchy from the root down, you can't just take a random joint and multiply by its parents world, if its parent world has not been transformed by its parent and so on back up to the root.

 

The same with your bones are they are in world space or local space related to their parent. Because their configuration will impact how your animations should be stored etc.

 

You will just have to work out the way your bones vs geometry is exported and adjust your code accordingly.

I would start with the simplest case draw it with no transforms. You could also write some code to draw your skeleton which can be useful to debug which aspect is broken.

 

And some configurations may be better than others in terms of performance etc.

But once its working in one configuration moving between them should be pretty straight forward.

Share this post


Link to post
Share on other sites

I would start by rendering some unskinned model just to make sure your data loading is correct.

 

Or just render the data your skinned geometry without the skinning transforms. You need to know if your vertices are in the world space or they have been saved after the bind pose has been removed (bone space/bind space?)

 

If the they are in world space, meaning if you draw them raw they look like the char in tpose or whatever, you usually will need to remove the bones bind pose first and then apply your animation transforms.

 

If they are not then drawing them raw they should look like a jumbled mess.

 

But my guess is that since you said you have the same bindShapeMatrix for every vert, the data could be in world space.

Otherwise you would need a bind pose transform for every joint to put them back in the correct spot. A single transform would just move the jumbled mess around.

 

For the parent joints make sure you are building the hierarchy from the root down, you can't just take a random joint and multiply by its parents world, if its parent world has not been transformed by its parent and so on back up to the root.

 

The same with your bones are they are in world space or local space related to their parent. Because their configuration will impact how your animations should be stored etc.

 

You will just have to work out the way your bones vs geometry is exported and adjust your code accordingly.

I would start with the simplest case draw it with no transforms. You could also write some code to draw your skeleton which can be useful to debug which aspect is broken.

 

And some configurations may be better than others in terms of performance etc.

But once its working in one configuration moving between them should be pretty straight forward.

 

Thank you for the reply.

 

First of all, if I render my model without applying any skinning transforms, it looks exacly how it should look like.

I am not exacly sure but I am guessing that they are in world space, since they are draw a the right place.

 

In the file, the bones are relative to their parent bone. So the matrix I read is the local matrix, relative to its parent.

To create the world matrix or a joint I take the parent world matrix and I multiply it by its local transform. I already start from the root bone, and move down the hierarchy. As for the root bone, its local transform becomes the world transform.

 

I have thought about trying to draw my skeleton to see if there is a problem, but I am still not an expert with openGL drawing stuff, but I will work on that.

 

Other than that, I have just no idea what else to try. I have switch matrices around to see the result, but nothing is as close as what I currently get.

Share this post


Link to post
Share on other sites

You basically have two options here

 

1. Your verts are in worldspace that means you need to transform them first by the original/binding bones pose.

So what you end up with is your verts in bone space. 

 

You would construct your bind skeleton in world space and then invert it.

Then transform all your verts with the inverted version of their bones using the correct weightings.

 

This will put them into the space of the bone space.

Then you can construct your new skeleton from animations etc in world space. Then transform the bone space verts by that skeleton and they should move back around your new skeleton.

 

So you basically subtracting out the bind skeleton then adding back in the animated skeleton.

If you do this correct you should be able to apply the bind skeleton back and it should leave the verts unchanged.

 

2. If you don't want to do this to your verts you can multiply the inverse of the bind skeleton to your animated skeleton to get the change in pose per bone and then apply that to your verts still in world space.

 

I usually do #1 so I can avoid the extra multiplication of the bind skeleton every time, since you only need to subtract out the bind skeleton once.

Share this post


Link to post
Share on other sites

You confused me a bit there. I am not actually sure what the bone space and world space.

The vertices I load from the collada file are in local space (I think).

Does that mean I can apply the inverse of the bone transform direcly to the vertices to put them into bone space. From there, I apply the new bone transform (im assuming there is only one bone here.) and ti would put them back into 'local space'.

 

Also, do you know what the bind shape matrix is for ? I read everywhere that you have to multiply the vertex with this bind shape matrix, and I am unsure what it is exacly for

 

EDIT

Now I think I understand a bit better. You need to apply the inverse of the bind pose in order to pring the vertex back to "0" then aply the current transformation matrix. Am I right ?

I can read an inverse bind matrix from the COLLADA, but shouldnt it be the same as if I take the absolute transform matrix of a joint in its bind pose and inverse it ? Because it isnt right now. I currently have only one bone in my model, and here is the data I read from the COLLADA file :

 

bone transformation matrix :

1.0    0.0                0.0                 0.0
0.0    7.54979E-8  1.0                  0.0
0.0    -1.0               7.54979E-8   0.0
0.0    0.0                0.0                 1.0
 
This is the manually inverted bone transformation matrix, not the one I read from the file:
1.0     -0.0                 0.0                 -0.0
-0.0    7.54979E-8    -1.0                  0.0
0.0     1.0                   7.54979E-8   -0.0
-0.0    0.0                  -0.0                 1.0
 
Wich is not the same as the corresponding inverse bind shape matrix, read from file:
1.0                      0.0                 0.0                    0.0
0.0                      0.0                 -1.0                   0.0
0.0                      1.0                 0.0                    0.0
-0.005478084     -0.5606597   -0.002059936   1.0
 
Should I ignore the inverted bind pose matrix read from the file and just create mine by inverting the absolute transform of the bone when loading it ?
Edited by yanjobin2641

Share this post


Link to post
Share on other sites

I got it working with a single bone affecting all vertices of a cube by 1. (!!)

I followed the whole process again starting from reading matrices in the collada file, and I realised there was something inversed in the translation part of the matrices. When building my matrices from a string (EX : 1 0 0 0 0 -1 5.88609e-8 0 0 -5.88609e-8 -1 0 0 0 0 1), the Y and Z translations were switched together. Thank you very much Eklipse your reply made me understand a lot of things that I thought I understood before. It is now much clearer how the matrices work together.

Share this post


Link to post
Share on other sites

Cool. Good to hear.

 

Understanding your data is very important. Otherwise you are just guessing what is going on. And build simple test cases in your modeling package and make sure they work.

So if you have it working with 1 vert then try 2 next.

 

Bone space is just the space after you remove the bones transform.

Basically the vertex positions are relative to the bone(s) they are attached too. Then you can just transform them by a new animated etc bone transform to put them back into world space or whatever space your skeleton is in.

 

I also almost mentioned that to make sure your transforms ordering is correct from the components.

Also make sure your matrices are multiplied in the correct order parent * child or child * parent when building your skeleton. Start with simple transforms so you can inspect it make sure the values look correct.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this