2. Animation
So let's think about what we need from FBX to make animation work in our renderer(game engine).
1. The skeleton hierarchy. Which joint is which joint's parent
2. For each vertex, we need 4 SkinningWeight-JointIndex pairs
3. The Bind pose matrix for each joint to calculate the inverse of global bind pose
4. The transformation matrix at time t so that we can transform our mesh to that pose to achieve animation
1. To get the skeleton hierarchy is pretty easy: basically we perform a recursive DFS from the root node of the scene
and we go down levels. If a node is of skeleton type, we add it into our list of joints and its index will just be the size of the list
Therefore, we can guarantee that the index of the parent is always gonna be less than that of the child
void FBXExporter::ProcessSkeletonHierarchy(FbxNode* inRootNode)
{
for (int childIndex = 0; childIndex < inRootNode->GetChildCount(); ++childIndex)
{
FbxNode* currNode = inRootNode->GetChild(childIndex);
ProcessSkeletonHierarchyRecursively(currNode, 0, 0, -1);
}
}
void FBXExporter::ProcessSkeletonHierarchyRecursively(FbxNode* inNode, int inDepth, int myIndex, int inParentIndex)
{
if(inNode->GetNodeAttribute() && inNode->GetNodeAttribute()->GetAttributeType() && inNode->GetNodeAttribute()->GetAttributeType() == FbxNodeAttribute::eSkeleton)
{
Joint currJoint;
currJoint.mParentIndex = inParentIndex;
currJoint.mName = inNode->GetName();
mSkeleton.mJoints.push_back(currJoint);
}
for (int i = 0; i < inNode->GetChildCount(); i++)
{
ProcessSkeletonHierarchyRecursively(inNode->GetChild(i), inDepth + 1, mSkeleton.mJoints.size(), myIndex);
}
}
2,3,4 Now we need to get the SkinningWeight-JointIndex pairs of each vertex. Unfortunately, my code is not very clean on animation so
the function below does steps 2,3,4 all at once. I will go over the code so please do not lose patience
Before seeing any code, please let me explain the terms used in FBXSDK. This is the part where I think most people find confusing because
FBXSDK's keywords do not match ours(game developer)
In FBX, there is such thing called a "Deformer". I see a deformer as a way to deform a mesh.
You know in Maya, you can have skeletal deformers but you can also have "contraints" to deform your mesh.
I think you can think of "Deformers" as the entire skeleton.
Inside each "Deformer"(I think usually a mesh only has one), you have "Clusters".
Each cluster is and is not a joint......
You can see a cluster as a joint, but actually, inside each cluster, there is a "link".
This "link" is actually the real joint, and it contains the useful information I need.
Now we delve into the code:
void FBXExporter::ProcessJointsAndAnimations(FbxNode* inNode)
{
FbxMesh* currMesh = inNode->GetMesh();
unsigned int numOfDeformers = currMesh->GetDeformerCount();
// This geometry transform is something I cannot understand
// I think it is from MotionBuilder
// If you are using Maya for your models, 99% this is just an
// identity matrix
// But I am taking it into account anyways......
FbxAMatrix geometryTransform = Utilities::GetGeometryTransformation(inNode);
// A deformer is a FBX thing, which contains some clusters
// A cluster contains a link, which is basically a joint
// Normally, there is only one deformer in a mesh
for (unsigned int deformerIndex = 0; deformerIndex < numOfDeformers; ++deformerIndex)
{
// There are many types of deformers in Maya,
// We are using only skins, so we see if this is a skin
FbxSkin* currSkin = reinterpret_cast<FbxSkin*>(currMesh->GetDeformer(deformerIndex, FbxDeformer::eSkin));
if (!currSkin)
{
continue;
}
unsigned int numOfClusters = currSkin->GetClusterCount();
for (unsigned int clusterIndex = 0; clusterIndex < numOfClusters; ++clusterIndex)
{
FbxCluster* currCluster = currSkin->GetCluster(clusterIndex);
std::string currJointName = currCluster->GetLink()->GetName();
unsigned int currJointIndex = FindJointIndexUsingName(currJointName);
FbxAMatrix transformMatrix;
FbxAMatrix transformLinkMatrix;
FbxAMatrix globalBindposeInverseMatrix;
currCluster->GetTransformMatrix(transformMatrix); // The transformation of the mesh at binding time
currCluster->GetTransformLinkMatrix(transformLinkMatrix); // The transformation of the cluster(joint) at binding time from joint space to world space
globalBindposeInverseMatrix = transformLinkMatrix.Inverse() * transformMatrix * geometryTransform;
// Update the information in mSkeleton
mSkeleton.mJoints[currJointIndex].mGlobalBindposeInverse = globalBindposeInverseMatrix;
mSkeleton.mJoints[currJointIndex].mNode = currCluster->GetLink();
// Associate each joint with the control points it affects
unsigned int numOfIndices = currCluster->GetControlPointIndicesCount();
for (unsigned int i = 0; i < numOfIndices; ++i)
{
BlendingIndexWeightPair currBlendingIndexWeightPair;
currBlendingIndexWeightPair.mBlendingIndex = currJointIndex;
currBlendingIndexWeightPair.mBlendingWeight = currCluster->GetControlPointWeights()[i];
mControlPoints[currCluster->GetControlPointIndices()[i]]->mBlendingInfo.push_back(currBlendingIndexWeightPair);
}
// Get animation information
// Now only supports one take
FbxAnimStack* currAnimStack = mFBXScene->GetSrcObject<FbxAnimStack>(0);
FbxString animStackName = currAnimStack->GetName();
mAnimationName = animStackName.Buffer();
FbxTakeInfo* takeInfo = mFBXScene->GetTakeInfo(animStackName);
FbxTime start = takeInfo->mLocalTimeSpan.GetStart();
FbxTime end = takeInfo->mLocalTimeSpan.GetStop();
mAnimationLength = end.GetFrameCount(FbxTime::eFrames24) - start.GetFrameCount(FbxTime::eFrames24) + 1;
Keyframe** currAnim = &mSkeleton.mJoints[currJointIndex].mAnimation;
for (FbxLongLong i = start.GetFrameCount(FbxTime::eFrames24); i <= end.GetFrameCount(FbxTime::eFrames24); ++i)
{
FbxTime currTime;
currTime.SetFrame(i, FbxTime::eFrames24);
*currAnim = new Keyframe();
(*currAnim)->mFrameNum = i;
FbxAMatrix currentTransformOffset = inNode->EvaluateGlobalTransform(currTime) * geometryTransform;
(*currAnim)->mGlobalTransform = currentTransformOffset.Inverse() * currCluster->GetLink()->EvaluateGlobalTransform(currTime);
currAnim = &((*currAnim)->mNext);
}
}
}
// Some of the control points only have less than 4 joints
// affecting them.
// For a normal renderer, there are usually 4 joints
// I am adding more dummy joints if there isn't enough
BlendingIndexWeightPair currBlendingIndexWeightPair;
currBlendingIndexWeightPair.mBlendingIndex = 0;
currBlendingIndexWeightPair.mBlendingWeight = 0;
for(auto itr = mControlPoints.begin(); itr != mControlPoints.end(); ++itr)
{
for(unsigned int i = itr->second->mBlendingInfo.size(); i <= 4; ++i)
{
itr->second->mBlendingInfo.push_back(currBlendingIndexWeightPair);
}
}
}
At the beginning I have this:
// This geometry transform is something I cannot understand
// I think it is from MotionBuilder
// If you are using Maya for your models, 99% this is just an
// identity matrix
// But I am taking it into account anyways......
FbxAMatrix geometryTransform = Utilities::GetGeometryTransformation(inNode);
Well, this is what I saw on the FBX SDK Forum. The officials there told us we should take into account
the "GeometricTransform". But according to my experience, most of the times, this "GeometricTransform"
is just an identity matrix. Anyways, to get this "GeometricTransform", use this function:
FbxAMatrix Utilities::GetGeometryTransformation(FbxNode* inNode)
{
if (!inNode)
{
throw std::exception("Null for mesh geometry");
}
const FbxVector4 lT = inNode->GetGeometricTranslation(FbxNode::eSourcePivot);
const FbxVector4 lR = inNode->GetGeometricRotation(FbxNode::eSourcePivot);
const FbxVector4 lS = inNode->GetGeometricScaling(FbxNode::eSourcePivot);
return FbxAMatrix(lT, lR, lS);
}
The very most most most most most most most most important thing in this code is how I get the
inverse of global bind pose of each joint. This part is very tricky and screwed up many people.
I will explain this in details.
FbxAMatrix transformMatrix;
FbxAMatrix transformLinkMatrix;
FbxAMatrix globalBindposeInverseMatrix;
currCluster->GetTransformMatrix(transformMatrix); // The transformation of the mesh at binding time
currCluster->GetTransformLinkMatrix(transformLinkMatrix); // The transformation of the cluster(joint) at binding time from joint space to world space
globalBindposeInverseMatrix = transformLinkMatrix.Inverse() * transformMatrix * geometryTransform;
// Update the information in mSkeleton
mSkeleton.mJoints[currJointIndex].mGlobalBindposeInverse = globalBindposeInverseMatrix;
So let's start from this GetTransformMatrix
The TransformMatrix is actually a legacy thing. It is the Global Transform of the entire mesh at binding time and all the clusters have exactly the same
TransformMatrix. This matrix would not be needed if your artists have good habits and before they rig the model, they "Freeze Transformations" on all channels
of the model. If your artists do "Freeze Transformations", then this matrix would just be an identity matrix.
Now we go on to GetTransformLinkMatrix. This is the very essence of the animation exporting code. This is the transformation of the cluster(joint) at binding time
from joint space to world space in Maya.
So now we are all set and we can get our inverse of global bind pose of each joint.
What we want eventually is the InverseOfGlobalBindPoseMatrix in "VertexAtTimeT = TransformationOfPostAtTimeT * InverseOfGlobalBindPoseMatrix * VertexAtBindingTime"
To get this, we do this: transformLinkMatrix.Inverse() * transformMatrix * geometryTransform
Now we are 2 steps away from animation. We need to get the SkinningWeight-JointIndex pair for each vertex and we still need to get the transformations
at different times in the animation
Let's deal with SkinningWeight-JointIndex pair first.
In our game engine, we have this relationship:
Vertex -> 4 SkinningWeight-JointIndex pairs
However, in FBXSDK the relationship is inverted.
Each cluster has a list of all the control points(vertices) it affects and how much it affects
The code below gets the relationship in the format we favor
But please recall that when I process control points, I stored all the control points into a map based on their indices.
This is where we can profit. With this map, here we can lookup and update the control point a cluster affect in O(1).
// Associate each joint with the control points it affects
unsigned int numOfIndices = currCluster->GetControlPointIndicesCount();
for (unsigned int i = 0; i < numOfIndices; ++i)
{
BlendingIndexWeightPair currBlendingIndexWeightPair;
currBlendingIndexWeightPair.mBlendingIndex = currJointIndex;
currBlendingIndexWeightPair.mBlendingWeight = currCluster->GetControlPointWeights();
mControlPoints[currCluster->GetControlPointIndices()]->mBlendingInfo.push_back(currBlendingIndexWeightPair);
}
Now we only the last piece in the puzzle: the Transformations at time T in the animation:
Note that this part is something I did not do well
My way is not very optimized since I get every keyframe
What should ideally be done is to get the keys and interpolate between them
But I guess this is a trade-off between space and speed
// Get animation information
// Now only supports one take
FbxAnimStack* currAnimStack = mFBXScene->GetSrcObject<FbxAnimStack>(0);
FbxString animStackName = currAnimStack->GetName();
mAnimationName = animStackName.Buffer();
FbxTakeInfo* takeInfo = mFBXScene->GetTakeInfo(animStackName);
FbxTime start = takeInfo->mLocalTimeSpan.GetStart();
FbxTime end = takeInfo->mLocalTimeSpan.GetStop();
mAnimationLength = end.GetFrameCount(FbxTime::eFrames24) - start.GetFrameCount(FbxTime::eFrames24) + 1;
Keyframe** currAnim = &mSkeleton.mJoints[currJointIndex].mAnimation;
for (FbxLongLong i = start.GetFrameCount(FbxTime::eFrames24); i <= end.GetFrameCount(FbxTime::eFrames24); ++i)
{
FbxTime currTime;
currTime.SetFrame(i, FbxTime::eFrames24);
*currAnim = new Keyframe();
(*currAnim)->mFrameNum = i;
FbxAMatrix currentTransformOffset = inNode->EvaluateGlobalTransform(currTime) * geometryTransform;
(*currAnim)->mGlobalTransform = currentTransformOffset.Inverse() * currCluster->GetLink()->EvaluateGlobalTransform(currTime);
currAnim = &((*currAnim)->mNext);
}
This part is pretty straightforward the only thing to be noted is that:
Maya currently does not support multi-take animations (Perhaps MotionBuilder does)
I will decide if I write about exporting materials based on how many people read this post
But it is pretty easy and can be learnt through "ImportScene" example
Conversions:
As we know, ideally Maya uses the same coordinate system as OpenGL does, which is (X-Right, Y-Up, Z-Out).
However, what if we want to use our animations in DirectX?
Basically
Position, Normal, Binormal, Tangent -> we need to negate the Z component of the Vector
UV -> we need to make V = 1.0f - V
Vertex order of a triangle -> change from Vertex0, Vertex1, Vertex2 to Vertex0, Vertex2, Vertex1(Basically invert the culling order)
Matrices:
1. Get translation component of the matrix, negate its Z component
2. Get rotation component of the matrix, negate its X and Y component
3. And we need to take the transpose