# FBX and Skinned Animation

This topic is 1287 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

Hi guys,

I'm trying to set up a skinned mesh keyframe animation system in my game. I create the mesh/bones in 3ds Max, export via FBX, then use the FBX SDK to get at the data to bring into my game.

I'm pretty sure what I need for a basic system is the following:

base data:
a) vertex positions of the mesh (in it's bind pose)
b) bone weights for each vertex
c) bone hierarchy
d) transform of each bone in it's bind pose

pose data:
a) for each pose, the local rotation of each bone (relative to it's parent).

Then I can smoothly go from one pose to another using interpolation (which I have working.) I'm not concerned with timing yet, I'll leave that for later.

I don't know how the files should be set up. Right now I have my bind pose mesh and bones in one file. Do I make a new file for each pose (or poses in a looped animation) ? Or maybe the poses need to be in the same file as the bind pose because transforms are relative to it ? Do I store each pose in a keyframe ? I'm lost.

I'm still learning how to extract data from FBX files and that's confusing me as well. I have more questions but I'll leave it here for now.

Any help/insight would be great.

##### Share on other sites

You may want to take a look at a couple articles - one on skinned mesh animation, the other on an animation controller for a skinned mesh. The first article outlines the various structures needed for a skinned mesh hierarchy and how such a hierarchy can be constructed. The latter provides some information on animation data, it's storage and use.

##### Share on other sites

There were several topics on this.

I've tried to create an article but because I'm a lazy bas*ard, the article isnt finished: http://www.gamedev.net/topic/646588-scene-graph-fbx-and-stuff/

It is a total crap(don't directly use it) but the basic idea is explaned.

To obtain all the keyframes and transforms for a node :

1) I use the AnimLayer to obtain the keyframes for the node. There MUST be a better way to do this!

2) Use the scene evaluator to obtain the *flattened* node transform for each key frame.

for each key frame do :
FbxAnimEvaluator* pEvaluator =  m_pFbxScene->GetEvaluator();
transformData = pEvaluator->GetNodeLocalTranslation/Rotation/Scale(CurrentNode, KeyFrameTime);

PS:

As said above:

 The FBX SDK on the other hand is a box of crazy and drives me insane.

Edited by imoogiBG

##### Share on other sites
Hi guys, thanks for the help. Things are starting to become clear but I still don't understand a lot. I'm thinking in terms of keys now (I was indeed getting confused looking at FbxPose stuff.)

I'm getting my bind pose data like this (pseudo):

...that seems to be working alright.

I don't understand the concept of FbxAnimStack and FbxAnimLayer. I think a stack is a collection of layers, and a layer is what used to be called a 'take' (which I think is like a key). I also don't understand what the FbxAnimEvaluator does.

I'm doing the modelling, rigging, and animating myself in 3ds Max. Let's say I want to do a simple two key idle animation for my soldier, I pose him at time=0 and at time=30. All I really need are the bone transforms for those two times (for my simple system.)

Can you put that in terms of FbxAnimStack, FbxAnimLayer, FbxAnimCurveNode, FbxAnimCurve, FbxAnimCurveKey ? I don't know where to look for the bone transforms at those two times.

Thanks again.

PS - Buckeye, your article on controllers is great, once I get the data from my FBX file I'll be referencing it a lot.
PSS - imoogiBG, I'm trying to go through your code but it's hard since some of it doesn't work in FBX 2015.1, especially the templated stuff. Edited by Endemoniada

##### Share on other sites

I'm getting my bind pose data like this (pseudo):

...that seems to be working alright.

The transform link is the way I do the bind pose matrix myself, so it looks correct to me.

I don't understand the concept of FbxAnimStack and FbxAnimLayer. I think a stack is a collection of layers, and a layer is what used to be called a 'take' (which I think is like a key). I also don't understand what the FbxAnimEvaluator does.

You can pretty much ignore the layers unless you intend to do some pretty advanced stuff, just collapse them at the start and you'll only have one layer per stack. (I don't remember the exact call, it's on the FbxAnimStack I believe.)

The FbxAnimStack is actually what used to be called a take as I remember it. For DCC tools which export multiple animations per FBX, you would have one of these stacks per animation. So, a file could have a walk stack, run stack, turn, jump etc. Maya seems to ignore this, I think Max will use them if you author the files in a specific manner.

Finally the evaluator is basically an animation playback system for the content of the FBX file. Basically if you set a stack as it's current context, it will allow you to sample the scene and get the transforms, etc from the scene at various times. This brings us to the rest:

I'm doing the modelling, rigging, and animating myself in 3ds Max. Let's say I want to do a simple two key idle animation for my soldier, I pose him at time=0 and at time=30. All I really need are the bone transforms for those two times (for my simple system.)

Can you put that in terms of FbxAnimStack, FbxAnimLayer, FbxAnimCurveNode, FbxAnimCurve, FbxAnimCurveKey ? I don't know where to look for the bone transforms at those two times.

The FBX SDK gives you a number of ways to get the data you might want. Unfortunately due to different DCC tools (Max/Maya/etc) you may not be able to get exactly the data you want. For instance, let's say you find the root bone and it has translation on it in the animation. You can access the transform in a number of ways. You can use the LclTransform property and ask for the FbxAMatrix at various times. Or you can call the evaluation functions with a time to get the matrix. Or you can use the evaluator's EvaluateNode function to evaluate the node at a time. And finally, the most complicated version is you can get the curve nodes from the properties and look at the curve's keys.

Given all those options, you might think getting the curves would be the way to go. Unfortunately Maya, for instance, bakes the animation data to a set of keys which have nothing to do with the keys actually setup in Maya. The reason for this is that the curves Maya uses are not the same as those FBX supports. So, even if you get the curves directly, they may have hundreds of keys in them since they might have been baked.

What this means is that basically unless Max has curves supported by FBX, it may be baking them and you won't have a way to find what the original two *poses* in your terms were. Generally you will iterate through time and sample the scene at a fixed rate. Yup, it kinda sucks and generally you'll want to simplify the data after sampling.

Hopefully this gets you past the understanding issues for the moment. FBX is fun stuff ain't it.

Edited by All8Up

##### Share on other sites

What this means is that basically unless Max has curves supported by FBX, it may be baking them and you won't have a way to find what the original two *poses* in your terms were. Generally you will iterate through time and sample the scene at a fixed rate. Yup, it kinda sucks and generally you'll want to simplify the data after sampling.

But that is what you want to do anyway.
There are several modes for interpolating between key frames and you want to reduce that down to all linear interpolations, because that is the only thing you should support at run-time.

You start by sampling the existing key frames into a set sorted by time, then you run over the animation and sample at fixed intervals of, say, 0.1666667 milliseconds (60 samples per second of animation).
Then you eliminate keys that are redundant when you linearly interpolate between the keys on both sides of it.

	/**
* Loads data from an FBX animation curve.
*
* \param _pfacCurve The curve from which to load keyframes.
* \param _pfnNode The node effected by this track.
* \param _ui32Attribute The attribute of that node affected by this track.
* \return Returns true if there are no memory failures.
*/
LSBOOL LSE_CALL CAnimationTrack::Load( FbxAnimCurve * _pfacCurve, FbxNode * _pfnNode, LSUINT32 _ui32Attribute ) {
SetAttributes( _pfnNode, _ui32Attribute );

LSUINT32 ui32Total = _pfacCurve->KeyGetCount();
static const FbxTime::EMode emModes[] = {
FbxTime::eFrames96,
FbxTime::eFrames60,
FbxTime::eFrames48,
FbxTime::eFrames30,
FbxTime::eFrames24,
};
LSUINT32 ui32FrameMode = 0UL;
if ( ui32Total ) {
while ( ui32FrameMode < LSE_ELEMENTS( emModes ) ) {
// Count how many entries we will add.
FbxTime ftTotalTime = _pfacCurve->KeyGetTime( ui32Total - 1UL ) - _pfacCurve->KeyGetTime( 0UL );
// We sample at emModes[ui32FrameMode] frames per second.
FbxLongLong fllFrames = ftTotalTime.GetFrameCount( emModes[ui32FrameMode] );
if ( ui32Total + fllFrames >= 0x0000000100000000ULL ) {
// Too many frames!  Holy crazy!  Try to sample at the next-lower resolution.
++ui32FrameMode;
continue;
}

m_sKeyFrames.AllocateAtLeast( static_cast<LSUINT32>(ui32Total + fllFrames) );
// Evaluate at the actual key times.
int iIndex = 0;
for ( LSUINT32 I = 0UL; I < ui32Total; ++I ) {
if ( !SetKeyFrame( _pfacCurve->KeyGetTime( I ), _pfacCurve->Evaluate( _pfacCurve->KeyGetTime( I ), &iIndex ) ) ) {
return false;
}
}
// Extra evaluation between key times.
iIndex = 0;
FbxTime ftFrameTime;
for ( FbxLongLong I = 0ULL; I < fllFrames; ++I ) {
ftFrameTime.SetFrame( I, emModes[ui32FrameMode] );
FbxTime ftCurTime = _pfacCurve->KeyGetTime( 0UL ) + ftFrameTime;
if ( !SetKeyFrame( ftCurTime, _pfacCurve->Evaluate( ftCurTime, &iIndex ) ) ) {
return false;
}
}

// Now simplify.
LSUINT32 ui32Eliminated = 0UL;
for ( LSUINT32 ui32Start = 0UL; m_sKeyFrames.Length() >= 3UL && ui32Start < m_sKeyFrames.Length() - 2UL; ++ui32Start ) {
const LSUINT32 ui32End = ui32Start + 2UL;
while ( m_sKeyFrames.Length() >= 3UL && ui32Start < m_sKeyFrames.Length() - 2UL ) {
// Try to remove the key between ui32Start and ui32End.
LSDOUBLE dSpan = static_cast<LSDOUBLE>(m_sKeyFrames.GetByIndex( ui32End ).tTime.GetMilliSeconds()) - static_cast<LSDOUBLE>(m_sKeyFrames.GetByIndex( ui32Start ).tTime.GetMilliSeconds());
LSDOUBLE dFrac = (static_cast<LSDOUBLE>(m_sKeyFrames.GetByIndex( ui32Start + 1UL ).tTime.GetMilliSeconds()) -
static_cast<LSDOUBLE>(m_sKeyFrames.GetByIndex( ui32Start ).tTime.GetMilliSeconds())) / dSpan;
// Interpolate by this much between the start and end keys.
LSDOUBLE dInterp = (m_sKeyFrames.GetByIndex( ui32End ).fValue - m_sKeyFrames.GetByIndex( ui32Start ).fValue) * dFrac + m_sKeyFrames.GetByIndex( ui32Start ).fValue;
LSDOUBLE dActual = m_sKeyFrames.GetByIndex( ui32Start + 1UL ).fValue;
LSDOUBLE dDif = ::abs( dInterp - dActual );
if ( dDif < 0.05 ) {
m_sKeyFrames.RemoveByIndex( ui32Start + 1UL );
++ui32Eliminated;
}
else {
// Move on to the next key frame and repeat.
break;
}
}
}
::printf( "\tOriginal key frames: %u\r\n\tFinal total key frames: %u %f\r\n", ui32Total,
m_sKeyFrames.Length(), m_sKeyFrames.Length() * 100.0f / static_cast<LSFLOAT>(ui32Total) );
break;
}
}
return ui32FrameMode < LSE_ELEMENTS( emModes );
}

/**
* Adds the given value at the given time to this track.
*
* \param _tTime The time of the keyframe to set.  If it already exists it will be overwritten.
* \param _fValue The value to set at the given time.
* \return Returns true if there was enough memory to complete the operation.
*/
LSE_INLINE LSBOOL LSE_CALL CAnimationTrack::SetKeyFrame( FbxTime _tTime, LSFLOAT _fValue ) {
LSFC_KEY_FRAME kfInsertMe = {
_tTime,
_fValue
};
return m_sKeyFrames.Insert( kfInsertMe );
}
		/** A key and value pair. */
typedef struct LSFC_KEY_FRAME {
/** The time of the keyframe. */
FbxTime								tTime;

/** The value at the time of the keyframe. */
LSFLOAT								fValue;
} * LPLSFC_KEY_FRAME, * const LPCLSFC_KEY_FRAME;
(This is not updated with the latest SDK.)

The algorithm for this is extremely trivial and the implementation is really just a handful of lines of code.
You will often end up with fewer key frames than there are in the animation as well, so even though you explode the number of key frames during the frequency sampling part, only the few key frames that matter actually remain, which means you may have even fewer than there were in the original Maya animation (but the same run-time result).

The fact is you need to do key frame reduction no matter what file format you are using. The Autodesk FBX SDK just makes it easy by giving you an animation evaluator.

So, yes, keys are the way to go.

L. Spiro Edited by L. Spiro

##### Share on other sites

osted Yesterday, 11:33 PM
AllEightUp, on 07 Jul 2014 - 10:21 PM, said:
What this means is that basically unless Max has curves supported by FBX, it may be baking them and you won't have a way to find what the original two *poses* in your terms were. Generally you will iterate through time and sample the scene at a fixed rate. Yup, it kinda sucks and generally you'll want to simplify the data after sampling.
But that is what you want to do anyway.
There are several modes for interpolating between key frames and you want to reduce that down to all linear interpolations, because that is the only thing you should support at run-time.

Yes, this is generally what you will want to do and I didn't mean to suggest it was a useless thing.  On the other hand, I would really like FBX to retain original key location information in the animation data so I can mark those keys as more important when evaluating key reductions.  The added context can greatly help when trying to preserve C1 continuity so things like fingers at the end of long bone chains don't jitter around so much.  It generally works out best if you keep those keys in preference to others since very often they are the extreme's of the curves and removal can throw off other reductions and increase the positional/velocity errors to a very notable degree.

##### Share on other sites

Hi guys,

I haven't been able to work on this again until this weekend. Thanks to your help I'm able to get the data from my FBX file but I'm still having difficulty getting it to work correctly. I think it's the offset (or inverse bind) transform that is tripping me up.

I created two bones, root and child, to test with. Here is how it looks in 3ds Max (plus my annotations):

Here is the data I pulled from the FBX:

lMatrix.GetT() : (0, 0, 0)
lMatrix.GetR() : (0, -90.0, 0) // note: I swap Y and Z

lMatrix.GetT() : (1.0, 0, 0)
lMatrix.GetR() : (0, 0, 0) // note: I swap Y and Z

At this point I need to create the offset for each bone, I did it by hand because it seemed straight-forward and I need to learn:

root: matrixRotationY(&root_bone.offset, halfPi); // the opposite of a -90 degree rotation, no transation

child: matrixTranslation(&child_bone.offset, -1.0, 0, 0); // the opposite of a (1.0, 0, 0) translation, no rotation

Is that correct ? It's not coming out right, take a look:

This is how it should look, it's the mesh rendered as is, without bones or transforms (the way it should look in it's bind pose):

I am using the identity for the local transforms (which should keep it in it's bind pose). I do it like this:


// bind to world
matrixRotationY(&bindToWorld, -halfPi); // because the root is rotated -90 degrees according to FBX

// fill in the transfrom matrices

// root bone
parent=bindToWorld; // the root bone's parent is the bindToWorld
matrixIdentity(&local); // using identity as noted above
matrixMultiply(&transforms[0], &local, &parent);

// child bone
parent=transforms[0]; // the root is the child's parent
matrixIdentity(&local); // using identity as noted above
matrixMultiply(&transforms[1], &local, &parent);

// now finalize by incorporating the offset matrix

matrixMultiply(&transforms[0], &root_bone.offset, &transforms[0]); // transforms[0] = rootOffset * transforms[0]
matrixMultiply(&transforms[1], &child_bone.offset, &transforms[1]); // transforms[1] = childOffset * transforms[1]



Note that for this test I simulate vertex weighting, the big "bone" would have a weight of 1.0 to the root, and the smaller "bone" would have a weight of 1.0 to the child. Maybe I am doing that part wrong but I don't think so. It's a single mesh like a regular skin would be.

As you can see from the image something isn't correct. I feel like I am almost there.

Thanks again for all your help.

##### Share on other sites

If I understand what you're trying to do, the offset matrix (used for skinning vertices, not display) is the inverse of the world transform for the child. Assuming left-to-right matrix multiplication is appropriate, the child's world transform is child_local_matrix * parent_world_matrix. It appears, in your case, the parent_world_transform = root_local_transform.

You've sort of got things backwards. The root local transform is the 90 deg rotation. The child local transform is the translation (1,0,0).

root_world_matrix = rotation by 90 deg;

child_local_matrix = translate by (1,0,0);

child_world = child_local_matrix * root_world_matrix;

child_offset = inverse(child_world);

The offset matrices are used in the vertex skinning process.

The "bind pose" for the child is simply it's world matrix.

Take another look at the article on skinned mesh animation I posted a link to earlier. You don't need to use the hierarchy described, but read about the process of animation near the end of the article. It discusses what the offset matrix is, how to calculate it and how to use it.

Edited by Buckeye

##### Share on other sites

I don't know what to do anymore so I'll give it another shot:

This is the data I get from my FBX file (scaling omitted):

shoulder (root):

T: (-2.0, 0, 0)

R: (0, 0, -180.0)

elbow:

T: (-6.0, 0, 0)

R: (0, 0, -180.0)

wrist:

T: (-10.0, 0, 0)

R: (0, 0, -90.0)

I am pretty sure those are the transformations when the skin was bound to the skeleton. It looks to me that there is no child-parent relationship, the data for each joint is global (in Model space.)

From what I've learned I need to compute the inverse bindpose matrix for each joint, this only has to be done once. Is the info above all I need to calculate those matrices ? How do I do that ?

***

Buckeye, I've read your article a few times and am still having trouble, I'm assuming the offset matrix is the same as the inverse bindpose matrix I'm trying to compute:

"offsetMatrix = MeshFrame.ToRoot * Inverse( bone.ToParent * parent.ToParent * ... * root.ToParent )"

"MeshFrame.ToRoot is the transform for moving the mesh into root-frame space" - is that the matrix that goes from Model space to the root joint's space ?

What is root.ToParent ? I thought roots don't have parents.

***

Also, considering that my data is not in a child-parent relationship will that equation still work ? Do I first have to put the bind pose in a child-parent relationship myself ?

I also noticed that when I get the data for the keyframes (poses) there is no child-parent relationship either. I use a parent-relative SRT for the joint poses, does that mean I have to construct child-parent relationships  myself ? That's kind of bugging me now.

I think the problem I've been having is that the tutorials I've read assume you can get the data exactly how it's supposed to be, and I'm fairly new to FBX as well as animation. I'm sorry if my questions are kinda dumb.

Thanks for helping.

##### Share on other sites

It's all pretty complicated, isn't it?

I'm assuming the offset matrix is the same as the inverse bindpose matrix I'm trying to compute:

Depends on what you're going to use the matrix for!   NOTE: The last set of data you posted looks like the origin-related orientation, and can be used to calculate the offset matrix - with one additional check: some modeling programs put the vertex data in a separate frame. That has to be accounted for, as mentioned in the section of that article "Back into Initialization - The Offset Matrix."

Modeling programs often export the pre-calculated offset matrix for each joint, in addition to the "to-parent" data. I'm not familiar with FBX, but, for instance, DirectX x-file exports provide "to-parent" transforms for each bone, and (separately) the offset matrices for each bone with the mesh data. Those mesh-data offset matrices include the mesh frame's "to-root" transform as mentioned above. It appears that the data you posted does NOT account for the mesh frame transform. However, IF the mesh-data is NOT in a separate frame (the data is relative to the origin) OR the mesh-frame transform is the identity matrix, then the data you posted can be used to calculate the offset matrix.

I think that's what you're referring to here:

"offsetMatrix = MeshFrame.ToRoot * Inverse( bone.ToParent * parent.ToParent * ... * root.ToParent )"

"MeshFrame.ToRoot is the transform for moving the mesh into root-frame space" - is that the matrix that goes from Model space to the root joint's space ?

IF "MeshFrame" is frame which parents the mesh data (the vertices), then that all appears to be correct. However, N.B., IF there are multiple sets of vertex data, that offset matrix is used ONLY for the vertex data for which it was calculated. If you only have ONE mesh in the model, that's fine. I mention that because I have several models with separate meshes for (e.g.) the head, the body, several weapons, a helmet, etc. There are separate sets of offset matrices for each mesh.

IF the data you posted above IS appropriate "to-root" data, then calculating the offset matrix using the above formula can be done using Inverse(bone.ToRoot) instead. Perhaps you should calculate the offsets as shown, and compare them to the inverse of the "to-root" data you posted.

What is root.ToParent ? I thought roots don't have parents.

The "root.ToParent" is what positions the root relative to the model origin. It's a fine distinction but important: the root's "to-root" is the identity matrix. The root's "to-parent" positions the root relative to the origin.

From the article:

// ... calculate all the Frame ToRoot matrices
CalcToRootMatrix( RootFrame, IdentityMatrix ); // the root frame has no parent - EDIT: Apologies. This may have confused you.
// I probably should have made it clear that FOR THE PURPOSES OF A TO-ROOT CALCULATION, the root should be considered
// not to have a parent, as the recursive routine uses "to-parent" matrices.


In the data you just posted:

shoulder (root):
T: (-2.0, 0, 0)
R: (0, 0, -180.0)


That's the root "to-parent" orientation as it positions the root relative to the origin.

As a refresher, skinned mesh animation is the process of rendering vertices (and the associated textured triangles) in a 3D world coordinate system. To do that, using the matrix approach mentioned in the article:

1. Move each vertex from model space into root space - that's what I'm assuming you're calling MeshFrame.ToRoot.

2. Move the vertex into bone space (joint/node space) - that's the bone's inverse-to-root matrix.

NOTE: steps 1 and 2 are combined in the offset matrix calculation you posted. Those matrices are calculated just once as they are "bind pose" related and the bind pose doesn't change.

The animation process then comprises calculations to move the vertex into 3D world space.

Have I sufficiently confused you?

Edited by Buckeye

##### Share on other sites

Hi again,

I can't for the life of me figure out why this isn't working. What I'm doing doesn't seem complicated at all, yet it will not work.

I am basing this off the last post by Buckeye in this thread:

http://www.gamedev.net/topic/589351-animation-bones-matrices-skeleton-how-does-it-work-/

I simplified everything and removed loops and recursions.

Here are the bones in their bind pose:

And here s the code:



// all rotations are done around the y-axis
float3 axis={0.0f, 1.0f, 0.0f};

// parent bone bind pose local matrix (relative to model space)
translation=matrixTranslation(0.0, 0.0, 0.0); // no translation
q=quaternionRotationAxis(axis, -45.0);
rotation=matrixRotationQuaternion(q);
parent_local=rotation*translation;

// child bone bind pose local matrix (relative to parent bone)
translation=matrixTranslation(2.0, 0.0, 2.0);
q=quaternionRotationAxis(axis, 45.0f);
rotation=matrixRotationQuaternion(q);
child_local=rotation*translation;

// parent bone inverse bind pose matrix
parent_bone.invbind=matrixInverse(parent_local);

// child bone inverse bind pose matrix
child_bone.invbind=matrixInverse(child_local*parent_local);



Alright, that's half the job. I compute those only once when I create my skeleton.

Now I'll make a pose (or keyframe) using SRT (actually RT since I have no scaling):



// parent bone pose will be same as the bind pose
parent_pose.translation={0.0, 0.0, 0.0}; // same as bind pose
q=quaternionRotationAxis(axis, -45.0); // same as bind pose
parent_pose.rotation=matrixRotationQuaternion(q);

// child bone pose will be no rotation relative to parent
child_pose.translation={2.0, 0.0, 2.0}; // same as bind pose
q=quaternionRotationAxis(axis, 0.0); // this is different from bind pose
child_pose.rotation=matrixRotationQuaternion(q);



And finally here is how I create the final (or skinning) matrices:

// parent bone final matrix
parent_slerp=parent_pose.rotation*parent_pose.translation; // this would normally be a slerp
parent_final=parent_bone.invbind*parent_slerp;

// child bone final matrix
child_slerp=child_pose.rotation*child_pose.translation; // this would normally be a slerp
child_final=child_bone.invbind*(child_slerp*parent_slerp);



And here it is, with the child bone not in the correct position (it is pointing in the correct direction though.)

What am I doing wrong ? This is killing me.

Thanks for everything.

##### Share on other sites

First: it's not clear how you are rendering the hierarchy. Consider how your bind pose should look. The root is rotated -45 and the child is rotated +45. The child should therefore be in line with the root. But it appears both have a local rotation of +45 (assuming left-hand rule).

Your calculation of the child inverse bind pose appears to be incorrect. The inverse bind pose for a frame/bone is with respect to the root. Your child inverse bind pose includes the root rotation-translation with respect to the world. You should use an identity matrix for the "root local" transformation in your child calculations.

You should not be calculating or using a root-frame "inverse bind pose." Considering the definition of "inverse bind pose" (a transformation from root-space to bone-space), at best, the root-frame "inverse bind pose" should be an identity matrix.

It appears that what you're calculating for the root "local" transform is really the world transform for the model.

Briefly, the sequence is:

1. for each child (NOT the root), calculate the inverse bind pose transform as a transformation from root-space (not world-space) to bone-space.

2. for each bone, calculate the local animation matrix. N.B., commonly the root frame is NOT animated!

// the root frame's parent.combined == identity matrix!

3. calculate the combined matrices for the hierarchy. I.e., bone.combined = bone.localAnimation * parent.combined;

4. calculate the final matrices for the hierarchy. I.e., bone.final = bone.invBindPose * bone.combined;

5. render the hierarchy.

Remember what you want the final rendering operation to be:

For each vertex, move the vertex position (which is relative to ROOT space) from root-space to bone-space, apply the bone animation transform to move the vertex from bone-space back to an animated position in root-space. Then apply the world-view-projection matrices. The world matrix is then how you want the root frame to rotate/translate.

I.e.,

vertexPos = vertexPos * boneInverseTransform [move to bone space] * finalBoneAnimationTransform [move to animated position in root space] * world [move from root space to world space] * view * projection.

Edited by Buckeye

##### Share on other sites

Buckeye, that really clears up a lot for me.

I was indeed including the root transform in my calculations, and I also didn't realize the root should be transformed by the world matrix.

I'm still having a problem though and I think it's related to your first observation:

"First: it's not clear how you are rendering the hierarchy. Consider how your bind pose should look. The root is rotated -45 and the child is rotated +45. The child should therefore be in line with the root. But it appears both have a local rotation of +45 (assuming left-hand rule)."

I really don't know what you mean by that. To me the rotation of the child in reference to the root being 45 deg makes it impossible to be in line (meaning they both point in the same direction.) I must not understand something.

Here is simplistic bind pose:

...with a hierarchy of pelvis->spine->neck. Notice the pelvis and spine are facing opposite directions and the spine and neck are facing the same direction.

pelvis.local=identity

spine.local=matrixRotationY(180.0); // rotate around y-axis, no translation

neck.local=matrixTranslation(0, 0, 2); // translation along z-axis, no rotation

Isn't that correct ? If it isn't then it's definitely my problem.

Thanks so much for helping, I'm going to have to get you an Amazon gift card or something :)

##### Share on other sites

You're correct about the root and the child not being inline. Apologies, my mistake. I was seeing (post #13) how both the root and the child appear to rotated about Y in the same direction (clockwise in the picture if the view is into the X-Z plane).

pelvis.local=identity
spine.local=matrixRotationY(180.0); // rotate around y-axis, no translation
neck.local=matrixTranslation(0, 0, 2); // translation along z-axis, no rotation

Isn't that correct ? If it isn't then it's definitely my problem.

ASSUMING you intend the spine to be the child of pelvis, and neck the child of spine, that appears correct.

##### Share on other sites

And this is what happens:

All I'm doing is changing the rotation for the neck from 0 to 45. It's pointing in the direction it's supposed to but it's translated wrong.

Here is my exact code for calculating the final matrices


// using the inverse bind as computed like this (see wireframe image in post 15):

skeleton.joints[0].invbind=matrixIdentity(); // pelvis
skeleton.joints[1].invbind=matrixInverse(matrixRotationY(pi)); // spine
skeleton.joints[2].invbind=matrixInverse(matrixTranslation(0.0f, 0.0f, 2.0f)); // neck

// pose T part

pose->joints[0].lclT=float3(0.0f, 0.0f, 0.0f);
pose->joints[1].lclT=float3(0.0f, 0.0f, 0.0f);
pose->joints[2].lclT=float3(0.0f, 0.0f, 2.0f);

// pose R part

quaternionIdentity(&pose->joints[0].lclR);
quaternionRotationAxis(&pose->joints[1].lclR, &axis, pi); // 180 deg about y-axis
quaternionRotationAxis(&pose->joints[2].lclR, &axis, pi*0.25f); // 45 deg about y-axis

//    bone chain

matrixIdentity(&transforms[0]);
parent=transforms[0];

for(int i=1;i<3;i++){

matrixTranslation(&translation,&pose->joints[i].lclT);
matrixRotationQuaternion(&rotation,&pose->joints[i].lclR);
matrixMultiply(&local,&rotation,&translation);
matrixMultiply(&transforms[i],&local,&parent);
parent=transforms[i];

}

//    finalize transformations

for(int i=1;i<3;i++){
matrixMultiply(&transforms[i],&skeleton.joints[i].invbind,&transforms[i]);
}



I've been working with this for a month and keep getting similar results no matter how I do it.

Maybe I'm just not cut out for this

##### Share on other sites

As a general approach, the code seems correct. It's a bit difficult to follow your changes as your variables change from post to post. So, things to look at:

Do you calculate the inverse bind matrix correctly?

Are you using the correct order of matrix multiplication? I.e., does your matrixMultiply function expect left-to-right or right-to-left order of the arguments?

When you set breakpoints and examine the matrices, are they correct?

How do render the scene?

If you use the same parameters for both bind and animation, do your final matrices==identity?

Maybe I'm just not cut out for this

You're working with one of the most difficult of intermediate subjects - skinned animation. If you've only been working at it a month, you're on schedule.

As a suggestion, pick names for your variables that are clear to you ... and to others if you're going to post code. Help others help you. E.g., you have an array named "transform" which you use for several different purposes. Why not use separate arrays, each named appropriately? It may prevent you from overwriting something unintentionally. That's the first thing I think of when I see code like that. For now, memory is cheap compared to the time you're spending, and the time it takes others to figure out what you're doing. Separate arrays will also allow you to compare results when you examine data. E.g., set a breakpoint; do a single calculation; is the output consistent with the input?

Edited by Buckeye

##### Share on other sites

Hey Buckeye, thanks a lot for the encouragement, it goes a long way.

I worked it out with pencil and paper and it seems to work like it should.

Now I'm working in a Windows program I made that will dump all the values in text to the screen, this way I can really see what's going on (It's easier to follow than breakpoints). To keep it simple I'm using two points to represent each bone.

I also switched to the D3DXMATRIX routines to eliminate the possibility that my routines are wrong.

I'll let you know what happens, and if I can't get it right I will use consistent terminology to show what I'm doing. I really feel like I understand this now but I'm still having some trouble implementing it. I think it's the matrix order.

Thanks for everything.

##### Share on other sites

Hi Buckeye,

I'm really at a loss. Either I am doing something fundamentally wrong, or the algorithm doesn't work. I did what you said and made everything clear, I also put everything in a single function that has no dependencies (except D3DX).

It should calculate the correct final (skinning) matrices. As a test I multiply the two blue points (left half of diagram) with joint_c.final, they should result in the two blue points on the right half of the diagram but they don't.

Here is the code:


struct Joint // or Bone or Frame
{
D3DXMATRIX offset; // pre-computed offset (or inverse bind)
D3DXMATRIX local; // transforms to parent's space (in bind pose)
D3DXMATRIX anim; // normally in SRT struct but here for simplicity
D3DXMATRIX combined; // anim*local * parent.anim*parent.local * ...(up to root)
D3DXMATRIX final; // final or skinning matrix
};

void Test()
{
D3DXMATRIX R,T,temp;

Joint joint_a; // root
Joint joint_b; // child of joint_a
Joint joint_c; // child of joint_b

// local transforms

D3DXMatrixIdentity(&joint_a.local);

D3DXMatrixRotationY(&R, -D3DX_PI*0.5f);
D3DXMatrixTranslation(&T, 2.0f, 0.0f, 0.0f);
joint_b.local=R*T;

D3DXMatrixTranslation(&T, 0.0f, 0.0f, 2.0f);
joint_c.local=T;

// offset (inverse bind)

temp=joint_a.local;
D3DXMatrixInverse(&joint_a.offset, NULL, &temp);

temp=joint_b.local*joint_a.local;
D3DXMatrixInverse(&joint_b.offset, NULL, &temp);

temp=joint_c.local*joint_b.local*joint_a.local;
D3DXMatrixInverse(&joint_c.offset, NULL, &temp);

// joint animation

D3DXMatrixIdentity(&joint_a.anim);
D3DXMatrixIdentity(&joint_b.anim);

// we only rotate (animate) joint_c with a 90deg rotation
D3DXMatrixRotationY(&joint_c.anim, D3DX_PI*0.5f);

// compute combined matrices

joint_a.combined=joint_a.anim*joint_a.local;

joint_b.combined=(joint_b.anim*joint_b.local)*joint_a.combined;

joint_c.combined=(joint_c.anim*joint_c.local)*joint_b.combined;

// final matrices

joint_a.final=joint_a.offset*joint_a.combined;
joint_b.final=joint_b.offset*joint_b.combined;
joint_c.final=joint_c.offset*joint_c.combined;

// transform (morph) some vertices

D3DXVECTOR3 va,vb;

va.x=2.0f; va.y=0.0f; va.z=2.0f;
vb.x=2.0f; vb.y=0.0f; vb.z=4.0f;

D3DXVec3TransformCoord(&va, &va, &joint_c.final);
D3DXVec3TransformCoord(&vb, &vb, &joint_c.final);
}



It's only about 20 lines of code with basic matrix math. I really don't know what I am doing wrong. If you don't want to go through my code maybe you can just look at the diagram and write some code the way you would do it.

I don't know what else to try at this point.

Thanks a lot.

##### Share on other sites

Your problem is in specifying the translation for joint_c. You have it translated along the Z axis of joint_b, which is apparently where you want it in WORLD (not local) coordinates. If you want joint_c to be "inline" with joint_b in the bind pose without regard to joint_b's rotation, you want to translate joint_c along joint_b's primary axis, which appears to be the X axis, not the Z. Joint_b's rotation will then rotate joint_c into the position shown on the left.

As questioned before, it's not clear what data you're using to render the images. You should show what you're actually getting, in addition to what you want. If joint_c is translated locally +2 along the Z-axis, then (by it's parent's orientation) rotated -pi/2 about the Y-axis to (-2, 0, 0), then translated +2 along the X-axis, it would appear at the origin, not (2,0,2) as you've shown. If joint_c is, instead, translated locally +2 along the X-axis, then it would be rotated to (0, 0, 2) and translated to (2, 0, 2).

Some tough love: In general, it's a lot easier to debug using actual data than "Here's what I want. My code doesn't work. What's wrong?"

By the way, in the code you've posted, you've still got re-used variables that makes it difficult to review, I'm afraid. To repeat, I would suggest you not re-use variables until (or even after) you've got the process well established. In this case, you're using bind-pose transforms to create animation transforms. Yeah, it appears to be "efficient" but, for review purposes, it requires going back and forth to check values and checking whether you've got trans*rot rather than rot*trans, etc.

The animation matrix (before combining) for each joint should be a local animation transform with respect to the joint's parent. It should reflect the animated orientation of the joint. However, you set the anim transform for joint_b to identity. Yeah, you're using the local transform to create an animation transform. But (I repeat) I'm suggesting you NOT REUSE variables until you get the process well understood. Later, when you create other animation sequences, you don't want to use bind-pose transforms mixed in with animation transforms.

For testing, it appears you want joint_b to be in the same orientation as it has in the bind pose. So code it again! Don't use the bind-pose transforms. For joint_b, in this particular situation, it may work, but setup the routine for the more general case.

Similarly, the local animation transform for joint_c should be the local rotation matrix times the local translation matrix. Don't use the bind-pose transforms. And, in this case, it appears you've coded joint_c translated along the WORLD Z-axis, instead of joint_b's local axis.

And.. in your D3DXVec3TransformCoord you re-use variables. I know that's for testing, but, eventually, in the shader, the input vertex will be transformed to an output vertex, not overwritten.

Edited by Buckeye

##### Share on other sites

c) bone hierarchy
d) transform of each bone in it's bind pose

pose data:
a) for each pose, the local rotation of each bone (relative to it's parent).

That is why FBX - keeping transformations derived relativly across binding rooting of a multi tree (like if it wanted to rule inverse procedural kinematics...lol) - is a one big mess stuff, creating a data hard to derive the final absolute matrix from. One needs to create absolute object space bone transformations per frame. FBX experimenatal unofficial uncompromised format with pile of SDKs end exporters, is too wrong for it.

Wite your own 3ds max script to output transformations of boning objects. Only viable solution for explicit animations and for rulling your production out. Trust me, I know what I am talking about.

##### Share on other sites

Hi Buckeye,

You pretty much solved my problem with your first paragraph. I was using the wrong axis for the local translation value. I also feel like I finally have a pretty good understanding of what it means to go from one bone space to another.

My goal was to setup a simple bone chain by hand so I could learn the mechanics of creating the offset (inverse bind) matrix as well as interpolating bones between poses to move the model vertices; and also to know exactly what data I should be getting from my FXB converter which brings me to this...

I know you don't use FBX but please bear with me. I can't figure out how to get the local transforms for each bone in the bind pose, or if they are even accessable. I can get the inverse world transforms of each bone and I already know the hiearchy. Can I do my own math to get the local bone transforms so I can compute the offset matrices myself ? If I can do that then I think I am all set.

I'm sorry if I got you a little annoyed with reusing variables and stuff, I will now be much more explicit in my code. It's like you said, it's better if I help others help me.

Thanks a lot man.

##### Share on other sites

I can get the inverse world transforms of each bone and I already know the hie[r]archy. Can I do my own math to get the local bone transforms so I can compute the offset matrices myself ? If I can do that then I think I am all set.

Maybe we have semantics problems. The inverse world transform of a bone is the offset matrix.

If you have the offset matrix (inverse world transform) and you can import animation matrices, you should be able to animate the mesh.

However, to answer your question, you should be able to extract local transforms if you have the hierarchy.

To keep things clear, hopefully, I'll define what I'm assuming.

The inverse world transform transforms from world space to bone space. The inverse of the inverse, therefore, is the bone's world transform, and transforms from bone space to world space.

That is, bone-world-xform = inverse( inverse-bone-world-transform ).

That bone-world-transform can be defined as:

A. bone-world-xform = bone-to-parent-matrix * parent-to-world-matrix.

To calculate just the bone-to-parent-matrix, you can transform the parent-to-world-matrix to an identity matrix by multiplying the parent-to-world by its inverse. That is, for any matrix, M * M-1 = Identity. Conveniently, you state you have the inverse world transforms. So, multiplying equation A by the inverse-parent-world-transform (the emphasized matrix multiplication below results in an identity matrix):

bone-world-xform * inverse-parent-to-world-transform = bone-to-parent-matrix * parent-to-world-matrix * inverse-parent-to-world-matrix.

bone-to-parent-matrix * identity = bone-to-parent-matrix, which I assume is what you mean by the bone's local transform.

In words, multiply a bone's to-world matrix by the inverse of the parent's to-world matrix to get the bone's to-parent matrix.

Is that what you'd like to do?

Edited by Buckeye