FBX and Skinned Animation

Started by
22 comments, last by Buckeye 9 years, 8 months ago

I don't know what to do anymore so I'll give it another shot:

bones2_zpsdcf63212.jpg

This is the data I get from my FBX file (scaling omitted):

shoulder (root):

T: (-2.0, 0, 0)

R: (0, 0, -180.0)

elbow:

T: (-6.0, 0, 0)

R: (0, 0, -180.0)

wrist:

T: (-10.0, 0, 0)

R: (0, 0, -90.0)

I am pretty sure those are the transformations when the skin was bound to the skeleton. It looks to me that there is no child-parent relationship, the data for each joint is global (in Model space.)

From what I've learned I need to compute the inverse bindpose matrix for each joint, this only has to be done once. Is the info above all I need to calculate those matrices ? How do I do that ?

***

Buckeye, I've read your article a few times and am still having trouble, I'm assuming the offset matrix is the same as the inverse bindpose matrix I'm trying to compute:

"offsetMatrix = MeshFrame.ToRoot * Inverse( bone.ToParent * parent.ToParent * ... * root.ToParent )"

"MeshFrame.ToRoot is the transform for moving the mesh into root-frame space" - is that the matrix that goes from Model space to the root joint's space ?

What is root.ToParent ? I thought roots don't have parents.

***

Also, considering that my data is not in a child-parent relationship will that equation still work ? Do I first have to put the bind pose in a child-parent relationship myself ?

I also noticed that when I get the data for the keyframes (poses) there is no child-parent relationship either. I use a parent-relative SRT for the joint poses, does that mean I have to construct child-parent relationships myself ? That's kind of bugging me now.

I think the problem I've been having is that the tutorials I've read assume you can get the data exactly how it's supposed to be, and I'm fairly new to FBX as well as animation. I'm sorry if my questions are kinda dumb.

Thanks for helping.

Advertisement

It's all pretty complicated, isn't it? biggrin.png


I'm assuming the offset matrix is the same as the inverse bindpose matrix I'm trying to compute:

Depends on what you're going to use the matrix for! wink.png NOTE: The last set of data you posted looks like the origin-related orientation, and can be used to calculate the offset matrix - with one additional check: some modeling programs put the vertex data in a separate frame. That has to be accounted for, as mentioned in the section of that article "Back into Initialization - The Offset Matrix."

Modeling programs often export the pre-calculated offset matrix for each joint, in addition to the "to-parent" data. I'm not familiar with FBX, but, for instance, DirectX x-file exports provide "to-parent" transforms for each bone, and (separately) the offset matrices for each bone with the mesh data. Those mesh-data offset matrices include the mesh frame's "to-root" transform as mentioned above. It appears that the data you posted does NOT account for the mesh frame transform. However, IF the mesh-data is NOT in a separate frame (the data is relative to the origin) OR the mesh-frame transform is the identity matrix, then the data you posted can be used to calculate the offset matrix.

I think that's what you're referring to here:


"offsetMatrix = MeshFrame.ToRoot * Inverse( bone.ToParent * parent.ToParent * ... * root.ToParent )"

"MeshFrame.ToRoot is the transform for moving the mesh into root-frame space" - is that the matrix that goes from Model space to the root joint's space ?

IF "MeshFrame" is frame which parents the mesh data (the vertices), then that all appears to be correct. However, N.B., IF there are multiple sets of vertex data, that offset matrix is used ONLY for the vertex data for which it was calculated. If you only have ONE mesh in the model, that's fine. I mention that because I have several models with separate meshes for (e.g.) the head, the body, several weapons, a helmet, etc. There are separate sets of offset matrices for each mesh.

IF the data you posted above IS appropriate "to-root" data, then calculating the offset matrix using the above formula can be done using Inverse(bone.ToRoot) instead. Perhaps you should calculate the offsets as shown, and compare them to the inverse of the "to-root" data you posted.


What is root.ToParent ? I thought roots don't have parents.

The "root.ToParent" is what positions the root relative to the model origin. It's a fine distinction but important: the root's "to-root" is the identity matrix. The root's "to-parent" positions the root relative to the origin.

From the article:


// ... calculate all the Frame ToRoot matrices
CalcToRootMatrix( RootFrame, IdentityMatrix ); // the root frame has no parent - EDIT: Apologies. This may have confused you.
// I probably should have made it clear that FOR THE PURPOSES OF A TO-ROOT CALCULATION, the root should be considered
// not to have a parent, as the recursive routine uses "to-parent" matrices.

In the data you just posted:


shoulder (root):
T: (-2.0, 0, 0)
R: (0, 0, -180.0)

That's the root "to-parent" orientation as it positions the root relative to the origin.

As a refresher, skinned mesh animation is the process of rendering vertices (and the associated textured triangles) in a 3D world coordinate system. To do that, using the matrix approach mentioned in the article:

1. Move each vertex from model space into root space - that's what I'm assuming you're calling MeshFrame.ToRoot.

2. Move the vertex into bone space (joint/node space) - that's the bone's inverse-to-root matrix.

NOTE: steps 1 and 2 are combined in the offset matrix calculation you posted. Those matrices are calculated just once as they are "bind pose" related and the bind pose doesn't change.

The animation process then comprises calculations to move the vertex into 3D world space.

Have I sufficiently confused you? cool.png

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

Hi again,

I can't for the life of me figure out why this isn't working. What I'm doing doesn't seem complicated at all, yet it will not work.

I am basing this off the last post by Buckeye in this thread:

http://www.gamedev.net/topic/589351-animation-bones-matrices-skeleton-how-does-it-work-/

I simplified everything and removed loops and recursions.

Here are the bones in their bind pose:

bones_004_zps403429a7.jpg

And here s the code:

 
 
// all rotations are done around the y-axis
float3 axis={0.0f, 1.0f, 0.0f};

// parent bone bind pose local matrix (relative to model space)
translation=matrixTranslation(0.0, 0.0, 0.0); // no translation
q=quaternionRotationAxis(axis, -45.0);
rotation=matrixRotationQuaternion(q);
parent_local=rotation*translation;

// child bone bind pose local matrix (relative to parent bone)
translation=matrixTranslation(2.0, 0.0, 2.0);
q=quaternionRotationAxis(axis, 45.0f);
rotation=matrixRotationQuaternion(q);
child_local=rotation*translation;

// parent bone inverse bind pose matrix
parent_bone.invbind=matrixInverse(parent_local);

// child bone inverse bind pose matrix
child_bone.invbind=matrixInverse(child_local*parent_local);
 
 

Alright, that's half the job. I compute those only once when I create my skeleton.

Now I'll make a pose (or keyframe) using SRT (actually RT since I have no scaling):

 

// parent bone pose will be same as the bind pose
parent_pose.translation={0.0, 0.0, 0.0}; // same as bind pose
q=quaternionRotationAxis(axis, -45.0); // same as bind pose
parent_pose.rotation=matrixRotationQuaternion(q);

// child bone pose will be no rotation relative to parent
child_pose.translation={2.0, 0.0, 2.0}; // same as bind pose
q=quaternionRotationAxis(axis, 0.0); // this is different from bind pose
child_pose.rotation=matrixRotationQuaternion(q);
 
 

And finally here is how I create the final (or skinning) matrices:

// parent bone final matrix
parent_slerp=parent_pose.rotation*parent_pose.translation; // this would normally be a slerp
parent_final=parent_bone.invbind*parent_slerp;

// child bone final matrix
child_slerp=child_pose.rotation*child_pose.translation; // this would normally be a slerp
child_final=child_bone.invbind*(child_slerp*parent_slerp);
 
 

And here it is, with the child bone not in the correct position (it is pointing in the correct direction though.)

bones_005_zps0ad3f54c.jpg

What am I doing wrong ? This is killing me.

Thanks for everything.

First: it's not clear how you are rendering the hierarchy. Consider how your bind pose should look. The root is rotated -45 and the child is rotated +45. The child should therefore be in line with the root. But it appears both have a local rotation of +45 (assuming left-hand rule).

Your calculation of the child inverse bind pose appears to be incorrect. The inverse bind pose for a frame/bone is with respect to the root. Your child inverse bind pose includes the root rotation-translation with respect to the world. You should use an identity matrix for the "root local" transformation in your child calculations.

You should not be calculating or using a root-frame "inverse bind pose." Considering the definition of "inverse bind pose" (a transformation from root-space to bone-space), at best, the root-frame "inverse bind pose" should be an identity matrix.

It appears that what you're calculating for the root "local" transform is really the world transform for the model.

Briefly, the sequence is:

1. for each child (NOT the root), calculate the inverse bind pose transform as a transformation from root-space (not world-space) to bone-space.

2. for each bone, calculate the local animation matrix. N.B., commonly the root frame is NOT animated!

// the root frame's parent.combined == identity matrix!

3. calculate the combined matrices for the hierarchy. I.e., bone.combined = bone.localAnimation * parent.combined;

4. calculate the final matrices for the hierarchy. I.e., bone.final = bone.invBindPose * bone.combined;

5. render the hierarchy.

Remember what you want the final rendering operation to be:

For each vertex, move the vertex position (which is relative to ROOT space) from root-space to bone-space, apply the bone animation transform to move the vertex from bone-space back to an animated position in root-space. Then apply the world-view-projection matrices. The world matrix is then how you want the root frame to rotate/translate.

I.e.,

vertexPos = vertexPos * boneInverseTransform [move to bone space] * finalBoneAnimationTransform [move to animated position in root space] * world [move from root space to world space] * view * projection.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

Buckeye, that really clears up a lot for me.

I was indeed including the root transform in my calculations, and I also didn't realize the root should be transformed by the world matrix.

I'm still having a problem though and I think it's related to your first observation:

"First: it's not clear how you are rendering the hierarchy. Consider how your bind pose should look. The root is rotated -45 and the child is rotated +45. The child should therefore be in line with the root. But it appears both have a local rotation of +45 (assuming left-hand rule)."

I really don't know what you mean by that. To me the rotation of the child in reference to the root being 45 deg makes it impossible to be in line (meaning they both point in the same direction.) I must not understand something.

Here is simplistic bind pose:

bones_10_zps3c4f06fa.jpg

...with a hierarchy of pelvis->spine->neck. Notice the pelvis and spine are facing opposite directions and the spine and neck are facing the same direction.

pelvis.local=identity

spine.local=matrixRotationY(180.0); // rotate around y-axis, no translation

neck.local=matrixTranslation(0, 0, 2); // translation along z-axis, no rotation

Isn't that correct ? If it isn't then it's definitely my problem.

Thanks so much for helping, I'm going to have to get you an Amazon gift card or something :)

You're correct about the root and the child not being inline. Apologies, my mistake. I was seeing (post #13) how both the root and the child appear to rotated about Y in the same direction (clockwise in the picture if the view is into the X-Z plane).


pelvis.local=identity
spine.local=matrixRotationY(180.0); // rotate around y-axis, no translation
neck.local=matrixTranslation(0, 0, 2); // translation along z-axis, no rotation

Isn't that correct ? If it isn't then it's definitely my problem.

ASSUMING you intend the spine to be the child of pelvis, and neck the child of spine, that appears correct.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

And this is what happens:

bones_11_zps35088b0c.jpg

All I'm doing is changing the rotation for the neck from 0 to 45. It's pointing in the direction it's supposed to but it's translated wrong.

Here is my exact code for calculating the final matrices


 
// using the inverse bind as computed like this (see wireframe image in post 15):

skeleton.joints[0].invbind=matrixIdentity(); // pelvis
skeleton.joints[1].invbind=matrixInverse(matrixRotationY(pi)); // spine
skeleton.joints[2].invbind=matrixInverse(matrixTranslation(0.0f, 0.0f, 2.0f)); // neck
 
// pose T part
 
pose->joints[0].lclT=float3(0.0f, 0.0f, 0.0f);
pose->joints[1].lclT=float3(0.0f, 0.0f, 0.0f);
pose->joints[2].lclT=float3(0.0f, 0.0f, 2.0f);

// pose R part
 
quaternionIdentity(&pose->joints[0].lclR);
quaternionRotationAxis(&pose->joints[1].lclR, &axis, pi); // 180 deg about y-axis
quaternionRotationAxis(&pose->joints[2].lclR, &axis, pi*0.25f); // 45 deg about y-axis
 
//    bone chain

matrixIdentity(&transforms[0]);
parent=transforms[0];

for(int i=1;i<3;i++){

 matrixTranslation(&translation,&pose->joints[i].lclT);
 matrixRotationQuaternion(&rotation,&pose->joints[i].lclR);
 matrixMultiply(&local,&rotation,&translation);
 matrixMultiply(&transforms[i],&local,&parent);
 parent=transforms[i];

}

 //    finalize transformations

 for(int i=1;i<3;i++){
  matrixMultiply(&transforms[i],&skeleton.joints[i].invbind,&transforms[i]);
 }
 

I've been working with this for a month and keep getting similar results no matter how I do it.

Maybe I'm just not cut out for this sad.png

As a general approach, the code seems correct. It's a bit difficult to follow your changes as your variables change from post to post. So, things to look at:

Do you calculate the inverse bind matrix correctly?

Are you using the correct order of matrix multiplication? I.e., does your matrixMultiply function expect left-to-right or right-to-left order of the arguments?

When you set breakpoints and examine the matrices, are they correct?

How do render the scene?

If you use the same parameters for both bind and animation, do your final matrices==identity?


Maybe I'm just not cut out for this

You're working with one of the most difficult of intermediate subjects - skinned animation. If you've only been working at it a month, you're on schedule.

As a suggestion, pick names for your variables that are clear to you ... and to others if you're going to post code. Help others help you. E.g., you have an array named "transform" which you use for several different purposes. Why not use separate arrays, each named appropriately? It may prevent you from overwriting something unintentionally. That's the first thing I think of when I see code like that. For now, memory is cheap compared to the time you're spending, and the time it takes others to figure out what you're doing. Separate arrays will also allow you to compare results when you examine data. E.g., set a breakpoint; do a single calculation; is the output consistent with the input?

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

Hey Buckeye, thanks a lot for the encouragement, it goes a long way.

I worked it out with pencil and paper and it seems to work like it should.

Now I'm working in a Windows program I made that will dump all the values in text to the screen, this way I can really see what's going on (It's easier to follow than breakpoints). To keep it simple I'm using two points to represent each bone.

I also switched to the D3DXMATRIX routines to eliminate the possibility that my routines are wrong.

I'll let you know what happens, and if I can't get it right I will use consistent terminology to show what I'm doing. I really feel like I understand this now but I'm still having some trouble implementing it. I think it's the matrix order.

Thanks for everything.

Hi Buckeye,

I'm really at a loss. Either I am doing something fundamentally wrong, or the algorithm doesn't work. I did what you said and made everything clear, I also put everything in a single function that has no dependencies (except D3DX).

It should calculate the correct final (skinning) matrices. As a test I multiply the two blue points (left half of diagram) with joint_c.final, they should result in the two blue points on the right half of the diagram but they don't.

bones_20_zpsab3e3a97.jpg

Here is the code:


 
struct Joint // or Bone or Frame
{
 D3DXMATRIX offset; // pre-computed offset (or inverse bind)
 D3DXMATRIX local; // transforms to parent's space (in bind pose)
 D3DXMATRIX anim; // normally in SRT struct but here for simplicity
 D3DXMATRIX combined; // anim*local * parent.anim*parent.local * ...(up to root)
 D3DXMATRIX final; // final or skinning matrix
};
 
void Test()
{
 D3DXMATRIX R,T,temp;

 Joint joint_a; // root
 Joint joint_b; // child of joint_a
 Joint joint_c; // child of joint_b

 // local transforms

 D3DXMatrixIdentity(&joint_a.local);

 D3DXMatrixRotationY(&R, -D3DX_PI*0.5f);
 D3DXMatrixTranslation(&T, 2.0f, 0.0f, 0.0f);
 joint_b.local=R*T;

 D3DXMatrixTranslation(&T, 0.0f, 0.0f, 2.0f);
 joint_c.local=T;

 // offset (inverse bind)

 temp=joint_a.local;
 D3DXMatrixInverse(&joint_a.offset, NULL, &temp);

 temp=joint_b.local*joint_a.local;
 D3DXMatrixInverse(&joint_b.offset, NULL, &temp);

 temp=joint_c.local*joint_b.local*joint_a.local;
 D3DXMatrixInverse(&joint_c.offset, NULL, &temp);

 // joint animation

 D3DXMatrixIdentity(&joint_a.anim);
 D3DXMatrixIdentity(&joint_b.anim);

 // we only rotate (animate) joint_c with a 90deg rotation
 D3DXMatrixRotationY(&joint_c.anim, D3DX_PI*0.5f);

 // compute combined matrices

 joint_a.combined=joint_a.anim*joint_a.local;

 joint_b.combined=(joint_b.anim*joint_b.local)*joint_a.combined;

 joint_c.combined=(joint_c.anim*joint_c.local)*joint_b.combined;

 // final matrices

 joint_a.final=joint_a.offset*joint_a.combined;
 joint_b.final=joint_b.offset*joint_b.combined;
 joint_c.final=joint_c.offset*joint_c.combined;

 // transform (morph) some vertices

 D3DXVECTOR3 va,vb;

 va.x=2.0f; va.y=0.0f; va.z=2.0f;
 vb.x=2.0f; vb.y=0.0f; vb.z=4.0f;

 D3DXVec3TransformCoord(&va, &va, &joint_c.final);
 D3DXVec3TransformCoord(&vb, &vb, &joint_c.final);
}
 

It's only about 20 lines of code with basic matrix math. I really don't know what I am doing wrong. If you don't want to go through my code maybe you can just look at the diagram and write some code the way you would do it.

I don't know what else to try at this point.

Thanks a lot.

This topic is closed to new replies.

Advertisement