FBX SDK skinned animation

Started by
39 comments, last by Dirk Gregorius 5 years, 6 months ago

While Dirk is searching his beautiful code you can try reading this code for becoming familiar with skeletal animation and skinning. However, this code ignores the geometric transform. Since most of my test models are from mixamo, I haven't needed such a transform. If you need this transform then simply transform the mesh vertices and normals using it, as Dirk pointed out.

Advertisement

Here is my cluster code


void RnCluster::Read( fbxsdk::FbxCluster* Cluster )
	{
	FbxCluster::ELinkMode LinkMode = Cluster->GetLinkMode();
	if ( LinkMode != FbxCluster::eNormalize )
		{
		return;
		}

	FbxNode* Link = Cluster->GetLink();
	if ( !Link )
		{
		return;
		}

	int ControlPointIndexCount = Cluster->GetControlPointIndicesCount();
	const int* ControlPointIndices = Cluster->GetControlPointIndices();
	const double* ControlPointWeights = Cluster->GetControlPointWeights();
	if ( ControlPointIndexCount <= 0 || !ControlPointIndices || !ControlPointWeights )
		{
		return;
		}

	FbxAMatrix TransformLinkMatrix;
	Cluster->GetTransformLinkMatrix( TransformLinkMatrix );
	mBindPose = RnMatrix4( TransformLinkMatrix );
	
	mVertexCount = ControlPointIndexCount;
	mVertexIndices.Resize( ControlPointIndexCount );
	mVertexWeights.Resize( ControlPointIndexCount );
	for ( int Index = 0; Index < mVertexCount; ++Index )
		{
		mVertexIndices[ Index ] = static_cast< int >( ControlPointIndices[ Index ] );
		mVertexWeights[ Index ] = static_cast< float >( ControlPointWeights[ Index ] );
		}
	}

 

Here is my mesh code (only control points for clarity)


void RnModelMesh::Read( FbxMesh* Mesh )
	{
	// Geometric transform
	FbxNode* Node = Mesh->GetNode();

	FbxVector4 GeometricTranslation = Node->GetGeometricTranslation( FbxNode::eSourcePivot );
	FbxVector4 GeometricRotation = Node->GetGeometricRotation( FbxNode::eSourcePivot );
	FbxVector4 GeometricScaling = Node->GetGeometricScaling( FbxNode::eSourcePivot );
	FbxAMatrix OffsetMatrix( GeometricTranslation, GeometricRotation, GeometricScaling );

	// Topology
	mTriangleCount = Mesh->GetPolygonCount();
	mTriangleIndices = static_cast<int*>( rnAlloc( 3 * mTriangleCount * sizeof( int ) ) );

	for ( int Index = 0; Index < mTriangleCount; ++Index )
		{
		mTriangleIndices[ 3 * Index + 0 ] = Mesh->GetPolygonVertex( Index, 0 );
		mTriangleIndices[ 3 * Index + 1 ] = Mesh->GetPolygonVertex( Index, 1 );
		mTriangleIndices[ 3 * Index + 2 ] = Mesh->GetPolygonVertex( Index, 2 );
		}

	// Vertex attributes
	mVertexCount = Mesh->GetControlPointsCount();
	mVertexPositions = static_cast<RnVector3*>( rnAlloc( mVertexCount * sizeof( RnVector3 ) ) );

	for ( int Index = 0; Index < mVertexCount; ++Index )
		{
		FbxVector4 ControPoint = OffsetMatrix.MultT( Mesh->GetControlPointAt( Index ) );
		mVertexPositions[ Index ] = RnVector3( ControPoint );
		}
	}

 

Unfortunately Dirk that code didnt seem to help because it seems to be what I already have in my code. My model is still skewed. Did you notice the edit I put into my last post on page 3 where I said I had fixed the skeletal problem with a switch back to the old code? What could that imply or is it unimportant?

 

There is probably a small transform issue. Personally I don't want to bake too many transforms into the bindpose. The reason is simply that I want to be able to draw the meshes for debugging without applying any transforms. So conceptually your mesh vertices need to be in model space and your bindpose transforms as well. That means you need to apply also the mesh node transform when you read the mesh vertices. I might have a mistake there since this transform is usually the identity matrix. But this is difficult to check without debugging. You can try this:

FbxAMatrix OffsetMatrix = Node->EvaluateGlobalTransform() * FbxAMatrix( GeometricTranslation, GeometricRotation, GeometricScaling );

I can try to put together a demo which assembles a simple skeleton from which you can extend. Not sure when I find time for this.

 

 

40 minutes ago, Dirk Gregorius said:

FbxAMatrix OffsetMatrix = Node->WorldTransform() * FbxAMatrix( GeometricTranslation, GeometricRotation, GeometricScaling );

 

FbxNode has no member WorldTransform()....?

 

If you do do a demo, that would be incredibly helpful. I find it really sad that there are no real tutorials on FBX converters at this stage of 3D. Its really holding me back and im sure a ton of other people too. But, dont just stop at the skeleton! Make a simple rectangle move like I have done complete with skinning. ?. I've been putting in the work, as seen in this thread, but its really really easy to get caught up in the details.

It is FbxNode::EvaluateGlobalTransform(). I fixed it in my earlier post.

I agree that there is indeed a lack of functional examples for animation. Most are way too complicated for such a conceptual easy problem in my opinion. The major obstacle is to understand the data you need and how to get it out of the modeling packages. This knowledge is available in game and movie companies, but is not shared very much. Probably because it is not the sexiest problem to talk about. Once you have the data, the actual algorithm is very easy. Either way, I see what I can do. 

Saying this, there is pretty good content on animation in general. I recommend to read the two references I posted earlier. In particular the free online course. You can even work through their assignments.

19 minutes ago, Dirk Gregorius said:

 

Saying this, there is pretty good content on animation in general. I recommend to read the two references I posted earlier. In particular the free online course. You can even work through their assignments.

Yes I've looked at those slides and links but its still hard for me to understand. The math notation is very difficult if not impossible for me to understand. I did very poorly in math when I was in school. I prefer code examples that I can compile to slides (as we probably all do)

I went back to Assimp and its working better than it did for me last year with the official master branch on github (about 1,500 commits later). I guess I should have tried it again sooner.

On 9/22/2018 at 1:54 PM, Dirk Gregorius said:

The next step is to deform the mesh using the new skeleton pose. This is where the skeleton/mesh binding comes in. This binding is often referred to as skin and it tells you how much a bone influences a vertex of the mesh. Again there are two ways to describe the data based on the association. E.g. either each bone knows which vertices it deforms or each vertex knows by which bone it is deformed. This looks something like this:



struct Cluster
{
  Bone* Bone;
	
  std::vector< int > VertexIndices;
  std::vector< float > VertexWeight;	
};


#define MAX_BONE_COUNT 4
  
struct Vertex
{
  Vector Position;
  Vector Normal;
  // ...
  
  int BoneIndex[ MAX_BONE_COUNT ];
  float BoneWeight[ MAX_BONE_COUNT ];
};

 

 

Now that I have Assimp working I think Assimp does the latter and my way with the XML does the former. How do I switch between the two? How do I go from each bone knowing which vertices it deforms to each vertex knowing by which bone it is deformed?

You can do something like this:

1) Go over each cluster and collect vertex influences (bone index and weight)

2) Go over each array of influences (per vertex) and sort, resize, and normalize

3) Copy influences into your vertex structure


struct VertexInfluence
{
	int BoneIndex = 0;
	float BoneWeight = 0.0f;
}

// Sort influence by bone weight
bool operator<( VertexInfluence Lhs, VertexInfluence Rhs )
{
	return Lhs.BoneWeight < Rhs.BoneWeight;           
}

// Normalize bone weights such that they sum up to 1
void Normalize( std::vector< VertexInfluence >& Influences )
{
   float TotalWeight = 0.0f;
   for ( const VertexInfluence& Influence : Influences ) 
   {
		TotalWeight += Influence.BoneWeight;
   }
  
   // This can happen if a vertex is not skinned
   if ( TotalWeight == 0.0f )
     return;
     
   for ( const VertexInfluence& Influence : Influences ) 
   {
		Influence.BoneWeight /= TotalWeight;
   }
}

// Collect vertex influences from clusters
std::vector< std::vector< VertexInfluences > > VertexInfluences;
VertexInfluences.resize( VertexCount );
for ( const Cluster* : mClusters )
{
    int BoneIndex = mBones.IndexOf( Cluster->Bone );
  
    for ( int i = 0; i < Cluster->VertexCount; ++i )
    {
       int VertexIndex = Cluster->VertexIndices[ i ];
       float VertexWeight = Cluster->VertexWeights[ i ];
      
       VertexInfluences[ VertexIndex ].push_back( { BoneIndex, VertexWeight } );
    }
}

// Process influences
for ( int i = 0; i < VertexCount; ++i )
{
   // For each vertex get the array of influences
   std::vector< VertexInfluence >& Influences = VertexInfluences[ i ];
   
   // Sort influences by weight (largest weights first)
   std::sort( Influences.begin(), Influences.end() );
  
   // Add zero weights if number of influence is smaller the needed or remove influences with smallest weights from back
   Influences.resize( MAX_BONE_COUNT );
  
   // Re-normalize weights such that they add up to 1
   Normalize( Influences );
}

 

HTH,

-Dirk

This topic is closed to new replies.

Advertisement