Sign in to follow this  
Pilpel

Skeletal animation in Assimp

Recommended Posts

I'm trying to learn how to do skeletal animation in Assimp and, as many people told me already, it's pretty harsh.

Until now I didn't really care about the order of the nodes and the respect to each node's parent. I just queried each node's mesh(es) and uploaded the required data (vertices, uvs, normals) to the GPU.

The result was looking good. I could even display models (that could also contain animations) in their bind pose without errors.

 

So, after attempting to learn skeletal animation I have a two questions:

1. Each node has a Matrix4x4 mTransformation member and I don't know why. Is it supposed to transform the vertices of the related meshes? Until now I didn't care about this data member at all.

2. Is there a good book that covers this subject widely? The (many) online tutorials I found are hard for me to understand.

Share this post


Link to post
Share on other sites

From the assimp docs: "mTransformation: The transformation relative to the node's parent."

 

The nodes (bones, frames, pick-a-name) form a hierarchy. The mTransformation is used to transform from the node's parent's space to the node's space in bind pose.

 

This article may give you a start in understanding skinned meshes (assuming that's where you're heading) and skeletal animation. In that article, the mTransformation you're asking about is called a "local transform."

 

From your description, it appears you're using only the transform for the node that "owns" a mesh. That's equivalent to using a world matrix to render a triangle. Skeletal animation of a skinned mesh is a whole new ballgame. That involves all the other nodes (without meshes) which represent the bones which will deform the mesh during animation.

 

You may want to look at Frank Luna's books Introduction to 3D Game Programming (he has one for DirectX 9 and one for DirectX 11). Also, Carl Granberg's Character Animation with Direct3D. I'm sure others will post their favorite books for various APIs (you don't mention what you're using for rendering.)

 

If you're not fully comfortable with matrices, and transformations from one space to another, you'll also need to spend some time brushing up on the concepts.

Share this post


Link to post
Share on other sites

From your description, it appears you're using only the transform for the node that "owns" a mesh.

What do you mean? I don't use the mTransformation matrix at all. How come I could display 3d models correctly until now if I haven't used it?

 

I render with OpenGL, btw.

Share this post


Link to post
Share on other sites

I could even display models (that could also contain animations) in their bind pose

 

 


What do you mean? I don't use the mTransformation matrix at all. How come I could display 3d models correctly until now if I haven't used it?

 

You said yourself the models appear in "bind pose." If you're not using the local matrix for rendering the mesh then it's not in bind pose, unless the local transform happens to be an identity matrix. If the local transform for the mesh is an identity matrix, that's a result of the modeling technique, not your rendering application. In future, you may work with models for which the local matrices are not identity, and/or for which the local matrix must be multiplied by a to-root matrix to appear as expected. Rendering the mesh vertices without transformation may result in something that looks close to what you expect, but you're not displaying it "correctly" with regard to your programming routines.

Edited by Buckeye

Share this post


Link to post
Share on other sites

I found this tutorial useful :

 

http://ogldev.atspace.co.uk/www/tutorial38/tutorial38.html

 

It shows how to render a skinned mesh from Assimp, using OpenGL.

 

Hope it helps smile.png

Didn't find that tutorial useful because its code relies on a system that was built in previous tutorials. The explanations weren't so good either. (that's just my reflection)

 

 


I could even display models (that could also contain animations) in their bind pose

 

 


What do you mean? I don't use the mTransformation matrix at all. How come I could display 3d models correctly until now if I haven't used it?

 

You said yourself the models appear in "bind pose." If you're not using the local matrix for rendering the mesh then it's not in bind pose, unless the local transform happens to be an identity matrix. If the local transform for the mesh is an identity matrix, that's a result of the modeling technique, not your rendering application. In future, you may work with models for which the local matrices are not identity, and/or for which the local matrix must be multiplied by a to-root matrix to appear as expected. Rendering the mesh vertices without transformation may result in something that looks close to what you expect, but you're not displaying it "correctly" with regard to your programming routines.

That's weird. I did manage to render the next (pretty complex) 3d model without caring about mTransformation.

Untitled.png

 

Anyway, I'm currently reading the tutorial you posted. I'll come back here if I have any questions.

Share this post


Link to post
Share on other sites

If you're rendering a model without worrying about the bone transformations, then you're not rendering an animated model. You're rendering a static model. That barbarian guy up there with his arms out in a T shape? Yeah, he's in his rest pose. That is how he was likely modeled from the start. If you want to render him as he picks up his axe and lops off a head, you're going to have to start worrying about those bone transformations because that's how it's done.

Share this post


Link to post
Share on other sites

If you want to render him as he picks up his axe and lops off a head, you're going to have to start worrying about those bone transformations because that's how it's done.

I thought they were "node transformations" and not "bone transformation"..?

Also, as Buckeye stated:

Rendering the mesh vertices without transformation may result in something that looks close to what you expect, but you're not displaying it "correctly" with regard to your programming routines.

It seems that even without worrying about animations, not using the mTransformation data member produces incorrect (?) results, and I was just lucky that it displays the model correctly?

Edited by Pilpel

Share this post


Link to post
Share on other sites

I thought they were "node transformations" and not "bone transformation"..?

 

 

The nodes (bones, frames, pick-a-name) ...

 

 

Also, from the article linked above:
 

What's a Bone? The term "frame" is used because it refers to a mathematical "frame of reference," or an orientation (SRT) with respect to (for instance) the world. The term "bone" is frequently used instead of "frame" because the concept can be used to simulate how a bone would cause the skin around it to move.

 

Add "node" to the list of synonyms. Assimp uses the term node.

 

 

I was just lucky that it displays the model correctly?


Yes. On the left - meshes without proper transforms. On the right - with proper transforms.

 

[attachment=26753:without-with.png]

Edited by Buckeye

Share this post


Link to post
Share on other sites

Still having problems with the matrices.

Am I supposed to multiply each node's mTransformation matrix by its parents' matrices, in order to get (and store) the "toRoot" matrix? cause that's what I'm doing, and I'm not sure if I'm getting correct results. (still static models)

Here's what I've done so far:

The main "Scene" class is called Model. It has a vector of Node objects, and each node has a vector of Mesh objects.

Each node also stores a glm::mat4 matrix (column-major) called toRoot. Some code:

 
//model.cpp

bool Model::import(const char *file)
{
	_scene = aiImportFile(file, aiProcess_Triangulate | 0/*aiProcess_FlipUVs*/);
	if(!_scene)
		aeError("could not load 3d model");

	return (_scene != nullptr);
}

void Model::load()
{
	if(_scene)
		_processNode(_scene->mRootNode, aiMatrix4x4()) //call _processNode with an identity matrix
}

void Model::draw()
{
	if(_scene)
		for(size_t i = 0; i < _nodes.size(); i++)
		{
			_shaderProgram.setUniform("ModelMatrix",  _transformation * _nodes[i].toRoot()); //_transformation is a private glm::mat4 that I use for scaling the model if it's too big.
			_nodes[i].draw();
		}
}

void Model::_processNode(aiNode *node, aiMatrix4x4 toRoot)
{
	_nodes.push_back( Node(node, _scene, toRoot, &_meshOptions) );

	for(unsigned int i = 0; i < node->mNumChildren; i++)
		_processNode(node->mChildren[i], node->mTransformation * toRoot); //aiMatrix4x4 is row-major, so I think this order of multiplication is correct
}


//node.cpp

Node::Node(aiNode *node, const aiScene *scene, aiMatrix4x4 toRoot, MeshOptions *meshOptions) : _node(node), _scene(scene)
{
	_aiMatrixToGlm(toRoot, _toRoot); //just inverses the give aiMatrix, and copies it to _toRoot.

	for(unsigned int i = 0; i < _node->mNumMeshes; i++)
	{
		unsigned int meshIndex = node->mMeshes[i];
		_meshes.push_back( Mesh(_scene->mMeshes[meshIndex], meshOptions) );
	}
}

void Node::draw()
{
	for(size_t i = 0; i < _meshes.size(); i++)
		_meshes[i].draw();
}

//private help function
void Node::_aiMatrixToGlm(const aiMatrix4x4 &aiMat, glm::mat4 &glmMat)
{
	//inverse from row-major to column-major
	aiMatrix4x4 temp = aiMat;
	temp.Inverse();

	for(int i = 0; i < 4; i++)
		for(int j = 0; j < 4; j++)
			glmMat[i][j] = temp[i][j];
}
 

in main.cpp, I first call Model::import(), then Model::load(), and then in the render function I call Model::draw().

The Mesh class constructor just uploads aiMesh data to the gpu using OpenGL functions. Mesh::draw() simply calls glDrawElements().

What am I doing wrong?

Edited by Pilpel

Share this post


Link to post
Share on other sites

Still having problems with the matrices. ... I'm not sure if I'm getting correct results ... What am I doing wrong?

 

You should take a look at this list of information regarding effective ways to get help with forum posts - in particular, the section titled "When asking about code."

 

After you've read that information, the next step in debugging is for you to identity the problem. So ... what problem have you identified?

Edited by Buckeye

Share this post


Link to post
Share on other sites

I noticed that when playing with the matrix calculations (in example changing the multiplication order, or simply omitting prog.setUniform() line) I get the same results. (the barbarian is located in the same spot everytime). This makes me think that my code is buggy.

So my question is if I'm doing this right, storing in each node a "toRoot" matrix which is the multiplication of the node mTransformation matrix and all of its parents' mTransformation matrices.

Edited by Pilpel

Share this post


Link to post
Share on other sites


... storing in each node a "toRoot" matrix which is the multiplication of the node mTransformation matrix and all of its parents' mTransformation matrices.

 

That description of the algorithm is correct. As you seem to be aware, the order of multiplication is important. That multiplication order includes scaling. I'm not particularly familiar with OGL or effects**, but understanding the difference between row-major and column-major is necessary.

 

** Is transposition of the matrix needed, and does "setUniform" do that?

 

If you're asking if your code implements the algorithm correctly, you should first attempt to determine that yourself. If there's a problem, you need to identify where in your code the problem occurs.

 

As with any program, both the code and the data must be correct for the results to be correct. When coding an algorithm for the first time, use the simplest possible data set to test it.

 

Skinned-mesh/skeletal-animation is an advanced technique. It appears you've written 100s of lines of code and may be using a fairly complex set of data imported using methods you don't fully understand. You should be prepared for that approach not to work.

Share this post


Link to post
Share on other sites

Add "node" to the list of synonyms. Assimp uses the term node.

Assimp also uses bones. (aiBone structure)

I remember struggling with assimp too initially.
It's been a few months since I last touched animations but aiBones only hold information such as inverse bind pose matrix but they call it offset matrix, bone indices & weights. The hierarchy is not represented at all here. That's where the aiNodes intervene. You have to construct a tree and match corresponding aiBones to aiNodes, using the names.

Share this post


Link to post
Share on other sites

So I got some stuff working now. I managed to display good looking (still static) models using the nodes' hierarchy and their toRoot matrix.

I've been trying for days to figure out the offset matrix purpose and how to generate one for every bone. (just so I know I got it right. Assimp generates an offset matrix for every bone automatically).

Please correct me if I'm wrong in any of the next statements:

1. From what I've read, the offset matrix transforms vertices from their mesh node space to their influence bone space. a simpler explanation: it transforms vertices from their node space to the bone's node space. By that, the offset matrix calculation should be like this:

//using the opengl matrix multiplication order (right is first, left is second)
offsetMatrix = Inverse(boneNode.toRoot()) * meshNode.toRoot(); 

2. Having the toRoot matrix for every node is enough in order to display a static model. No other data like the offset matrix is needed.

 

Regarding the first statement, I checked if the offset matrix calculated by Assimp is equal to the offset matrix I calculate with my approach. I failed.

Here's my attempt:

aiBone *bone = mesh->mBones[i];
			 
cout << "bone->mName.C_Str(): " << bone->mName.C_Str() << endl;
cout << "bone->mOffsetMatrix:\n";
disp(bone->mOffsetMatrix); //displays a given matrix

aiNode *boneNode = findNodeByName(bone->mName.C_Str());
if(boneNode == nullptr)
	cout << "findNodeByName(bone->mName.C_Str()) is nullptr\n";
else {
	aiMatrix4x4 nodeToRoot = calcToRoot(meshNode);
	aiMatrix4x4 boneToRoot = calcToRoot(boneNode);

	cout << "my calculated offset matrix:\n";
	disp(boneToRoot.Inverse() * nodeToRoot);
}

Here's the whole piece of code, if it helps. (not too long)

Note that I also tried playing with the matrices multiplication order. offsetMatrix is still not close to be the assimp's bone offset matrix.

 

Help would be greatly appreciated!

Edited by Pilpel

Share this post


Link to post
Share on other sites

(The offset matrix) transforms vertices from their node space to the bone's node space

 

The description is correct. However, it may be that the assimp offset matrix is just boneToRoot.Invervse(). Have you made that comparision?

 

In some file formats, that, in fact, is the way it's done, and you have to do the meshToRoot multiplication yourself, particularly if there is only one set of bone offset matrices in the file. The reason for that is an animated model may have several meshes, each with its own ToRoot. Consider, if there are multiple meshes in the file, either there would have to be a set of offset matrices per mesh, or just one set of offsets and the user would apply the individual meshToRoot matrices separately.

 

EDIT:

 

 


2. Having the toRoot matrix for every node is enough in order to display a static model. No other data like the offset matrix is needed.

 

That is often the case, but there is no standard for how data must be applied. I.e., the only rule is: there ain't no rules. If using only toRoot mats for a static model works for that file, then the only thing that can be said is: it works for that file.

 

I've run across some files which were obviously intended for a particular implementation. E.g., IIRC, I have run into cases where separate meshes in the file (such as a weapon, or a helmet) were origin based. The user of the file was expected to attach that mesh as desired. Also, I ran across a model in which several animation sequences were all crammed into a single animation set, and the individual sequences were intended to be invoked by a range of ticks (frame times). I.e., frame 0 was base pose; frames 1-10 were Walk, frames 11-32 were RaiseAxe, etc., etc. That frame time information was not included in the file.

Edited by Buckeye

Share this post


Link to post
Share on other sites

The description is correct. However, it may be that the assimp offset matrix is just boneToRoot.Invervse(). Have you made that comparision?

I tried comparing the matrices and they are still different. Can anyone who worked with Assimp before confirm what exactly does assimp's offsetMatrix do? It isn't equal to any matrix transformation I tested till now.

Edited by Pilpel

Share this post


Link to post
Share on other sites

Two more questions regarding the "final matrix":

 

1. Let's say that assimp's offset matrix for each bones transforms from root node space to bone space ( Inverse(boneToRoot) ). The final matrix for each bone in the render cycle should be the following: Inverse(offsetMatrix) * animationMatrix * offsetMatrix * nodeToRoot. In order words:

Mesh node space -> root node space,

root node space -> bone node space,

apply the animation matrix, so: bone node space -> animated bone node space.

now back to root space: animated bone node space -> (animated) root node space.

Is this correct?

 

2. Many people suggest to extract only the "bone nodes" from the node hierarchy and upload them to a Skeleton object which contains only bones.

When calculating the final matrix for each bone (the only that will be uploaded to the shader), should I care about bones only or should I care about nodes (that might be under the bones in the node hierarchy) too?

What's the preferred way of doing this?

Edited by Pilpel

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this