• Create Account

## ASSIMP skinned mesh with DX9 problem

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

12 replies to this topic

### #1IceBreaker23  Members

Posted 06 March 2014 - 04:14 AM

Hello!

With the help of the forum I managed to fix my skinning shader problem. Now I have another problem.

I am using ASSIMP and DirectX9. I implemented the skeletal animation according to this tutorial: http://ogldev.atspace.co.uk/www/tutorial38/tutorial38.html

If I use the default pose(T-pose) the model is shown just how it should be(see attached screenshot model_tpose.png).

As soon as I use the animations from animation frame0 it goes all weird(see attached screenshot model_frame0.png).

I think that something with calculating the NodeTransformation is wrong.

Here is the code of the update function called each frame(the Matrix class is from SimpleMath.h, an oop wrapper for DirectXMath.h):

void SkinnedMesh::UpdateNode(aiAnimation *animation,float animationTime, aiNode *node, Matrix parentTransform)
{
const char *nodeName = node->mName.C_Str();
const aiNodeAnim *nodeAnim = FindNodeAnim(animation, nodeName);

Matrix NodeTransformation(*(Matrix *)&node->mTransformation);

//if I comment the whole if-section then it works, because the bones are then in default pose
if(nodeAnim != NULL)
{
//somewhere here should be the problem
aiVector3D Scaling = nodeAnim->mScalingKeys[0].mValue;
Matrix ScalingM = Matrix::CreateScale(Scaling.x,Scaling.y,Scaling.z);

aiQuaternion q = nodeAnim->mRotationKeys[0].mValue;
Matrix RotationM = Matrix::CreateFromQuaternion(Quaternion(q.x,q.y,q.z,q.w));;

aiVector3D Translation = nodeAnim->mPositionKeys[0].mValue;
Matrix TranslationM = Matrix::CreateTranslation(Translation.x,Translation.y,Translation.z);

NodeTransformation = TranslationM * RotationM * ScalingM;
}

Matrix finalTransform = parentTransform * NodeTransformation;

if(_boneMap.find(nodeName) != _boneMap.end())
{
int boneID = _boneMap[nodeName];
_finalBoneTransformations[boneID] = _globalInverseMatrix * finalTransform *  this->_bones[boneID].offsetMatrix;
}

for(unsigned int i = 0;i<node->mNumChildren;i++)
{
UpdateNode(animation,animationTime,node->mChildren[i],finalTransform);
}

}


I hope someone can help me and tell me why it does that.

Btw: the problem is likely not right-handed/left-handed matrices problem, since ASSIMP and SimpleMath both use right-handed matrices.

### #2Buckeye  GDNet+

Posted 06 March 2014 - 08:21 AM

I think that something with calculating the NodeTransformation is wrong.

If the pose position is fine (in the left image) when you comment out the section you indicate, that seems likely. But see my comment below about the Matrix NodeTransformation line.

I haven't made the transition to DirectXMath yet, but, just on inspection, the form of the calculation in if( nodeAnim != NULL) {...} looks good. Normally I would recommend (as with your shader problem): if the code looks good, check the input/output values. However, that's not quite as easy here because it's a bit difficult to know what the correct values should be. Also, because you're doing calcs for the entire frame hierarchy, there's more than just one value to check.

I'm not familiar with how C# works under the hood, so I'm just trying to understand what everything does. I'm a bit curious about this line:

Matrix NodeTransformation(*(Matrix *)&node->mTransformation);


Does it copy mTransformation or create a reference to it? If it's just a copy, that's fine. My concern is that you may be storing the calculated animation transform back in the node. If you're not sure, you can check by visually examining node->mTransformation before and after the animation calculation. node->mTransformation should not have been modified. I understand that you set up the code to display the pose position for testing, but normally you shouldn't need node->mTransformation to calculate the final transforms.

If you're absolutely positive that NodeTransformation is just a copy, then you might want to try to isolate the problem to a single value. First, the recommendation and then the explanation.

Pick a node in your frame hierarchy very high in the tree - like an arm or a hand. Then use something like the following, where "Left_Hand" is an actual bone name in your structure:

if( nodeAnim != NULL && nodeName.c_str() == "Left_Hand")
{
....
}

What that should do is render most of the model in pose position and apply the animation to only the hand and its children. If the pose position parts look good, then the problem must be related to calculating the animation transform, retrieving the animation transform [ FindNodeAnim(animation, nodeName) ], accessing the key values or creating the animated NodeTransformation.

It's a start.

Edited by Buckeye, 06 March 2014 - 08:29 AM.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

### #3IceBreaker23  Members

Posted 06 March 2014 - 09:24 AM

For a start it's not C#

I changed the line

Matrix NodeTransformation(*(Matrix *)&node->mTransformation);

to

Matrix NodeTransformation = Matrix(*(Matrix *)&node->mTransformation);


because I wasn't sure about this constructor call as well. I checked and the matrix-constructor takes a const value.(so no modification possible)

I limited the if(nodeAnim != NULL) to only work with "thumb.L". The result is attached as a screenshot.

EDIT: the object beside the hand is a lantern.

#### Attached Thumbnails

Edited by IceBreaker23, 06 March 2014 - 09:26 AM.

### #4Buckeye  GDNet+

Posted 06 March 2014 - 10:24 AM

It's not C#. Well, there you go. As mentioned, I'm not familiar with C# and proved the point!

EDIT: my first try at this post was total garbage and I deleted it. Back to the drawing board for a moment.

First: from what you've done, it definitely looks like the problem is related to accessing the animation keys, doing calcs with the keys, and using the result for the final transform.

I don't know what some of the functions do.

What does FindNodeAnim(animation, nodeName) do? I.e., what's the nodeAnim object?

In nodeAnim->mScalingKeys[0].mValue, what is the subscript "[0]" ?

Are you accessing the animation timed key-frames directly?

You don't use input animationTime. Are you just doing calcs for time = 0 for testing purposes?

You can check do a bit of a check on actual values. Many animations use only rotations. If that's true in this case, the scale key should be ( 1, 1, 1) and the translate key should be the child's position with respect to the parent and not likely to change. That is, the animation translate key for the thumb should be the same as the pose position translate key. Don't know how you might check that if mTransformation is being reused for animation calcs.

EDIT2:

aiQuaternion q = nodeAnim->mRotationKeys[0].mValue;
Matrix RotationM = Matrix::CreateFromQuaternion(Quaternion(q.x,q.y,q.z,q.w));;

Just looking: why the double ";;" ?

Also, don't know the requirements for the various functions, but can you create the matrix from mValue directly? or from q itself?

EDIT3: Note - this approach is pretty much the same as with your shader problem. Be sure you know what the code should be doing and is coded to do that; check that the values going coming into the calc are correct; check that the resulting values are correct. Somewhere along that line is the problem.

Edited by Buckeye, 06 March 2014 - 11:18 AM.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

### #5IceBreaker23  Members

Posted 06 March 2014 - 11:29 AM

The function FindNodeAnim looks in all the animation's channels to find the respective aiAnimNode* to the bone-name.

Here is the function FindNodeAnim:

aiNodeAnim *SkinnedMesh::FindNodeAnim(aiAnimation *animation,const char *nodeName)
{
for(unsigned int i = 0;i<animation->mNumChannels;i++)
{
if(strcmp(nodeName,animation->mChannels[i]->mNodeName.C_Str()) == 0)
{
return animation->mChannels[i];
}
}

return NULL;
}


For testing purposes I only access time = 0(so I dont have the interpolation code in as well, which would make the whole debugging even more complex)

The double ";;" was just a typo and I removed it.

Part of the problem is that there are 2 libraries I need to convert between. One is the ASSIMP-math library(respective data types are aiQuaternion,aiVector3) and the SimpleMath library(http://blogs.msdn.com/b/shawnhar/archive/2013/01/08/simplemath-a-simplified-wrapper-for-directxmath.aspx)

I can also create an aiMatrix4x4 from the aiQuaternion and then cast it to my Matrix-type. I actually tried it and it gives same result.

I tried to isolate the rotation as you suggested. Now the code is:

void SkinnedMesh::UpdateNode(aiAnimation *animation,float animationTime, aiNode *node, Matrix parentTransform)
{
const char *nodeName = node->mName.C_Str();
const aiNodeAnim *nodeAnim = FindNodeAnim(animation, nodeName);

Matrix NodeTransformation = Matrix(*(Matrix *)&node->mTransformation);

if(nodeAnim != NULL && strcmp(nodeName,"thumb.L") == 0)
{
aiQuaternion q = nodeAnim->mRotationKeys[0].mValue;
Matrix RotationM = Matrix::CreateFromQuaternion(Quaternion(q.x,q.y,q.z,q.w));

NodeTransformation = Matrix::CreateTranslation(NodeTransformation.m[0][3],NodeTransformation.m[1][3],NodeTransformation.m[2][3]) * RotationM * Matrix::CreateScale(1.0f);
}

Matrix finalTransform = parentTransform * NodeTransformation;

if(_boneMap.find(nodeName) != _boneMap.end())
{
int boneID = _boneMap[nodeName];
_finalBoneTransformations[boneID] = _globalInverseMatrix * finalTransform *  this->_bones[boneID].offsetMatrix;
}

for(unsigned int i = 0;i<node->mNumChildren;i++)
{
UpdateNode(animation,animationTime,node->mChildren[i],finalTransform);
}
}


The output is the same as in the previously posted screenshot with the weird left thumb.

EDIT: I know its something similar to my shader problem. The only problem is that I do not know the math behind quaternions and so checking values doesnt help very much.

Edited by IceBreaker23, 06 March 2014 - 11:35 AM.

### #6Buckeye  GDNet+

Posted 06 March 2014 - 11:52 AM

Looks like you're in a similar situation as with your shader problem. If the code looks right, you have to check the values.

I don't know if your import file is in text or binary format. If it's in text, find key-frame 0 for the thumb and see if that's what you're getting from the mValues. If the import is binary, that's a bit more difficult. As I noted in one of my edits above, you can at least check if the mValues are reasonable.

For instance, given that the thumb's "to-parent" transform should keep it close to its parent, it looks like it's getting a pretty big translate or scale component. Is that's what coming into the update function, or out from it? You don't show the front view of the "weird" thumb. Is it's position relative to the hand strangely the same as the distance it should be from the root node?

You assume the key-frame data for a single node (thumb, in this case) is a local transform, i.e., with respect to its parent. That's how you use the data. Is that a correct assumption?

Edited by Buckeye, 06 March 2014 - 11:54 AM.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

### #7IceBreaker23  Members

Posted 06 March 2014 - 12:46 PM

I just checked frame0 in my .md5anim file and compared it. The translation as well as the rotation values that I get from mValue are correct and reasonable(trans(0.054595 0.018457 -0.015135), rot_in_quaternion(0.176653 -0.062347 -0.335345))

I got this model from the tutorial(http://ogldev.atspace.co.uk/www/tutorial38/tutorial38.html) and also just translated part of the code into my own class.

Here is the original update function of the tutorial:

void Mesh::ReadNodeHeirarchy(float AnimationTime, const aiNode* pNode, const Matrix4f& ParentTransform)
{
string NodeName(pNode->mName.data);

const aiAnimation* pAnimation = m_pScene->mAnimations[0];

Matrix4f NodeTransformation(pNode->mTransformation);

const aiNodeAnim* pNodeAnim = FindNodeAnim(pAnimation, NodeName);

if (pNodeAnim) {
// Interpolate scaling and generate scaling transformation matrix
aiVector3D Scaling;
CalcInterpolatedScaling(Scaling, AnimationTime, pNodeAnim);
Matrix4f ScalingM;
ScalingM.InitScaleTransform(Scaling.x, Scaling.y, Scaling.z);

// Interpolate rotation and generate rotation transformation matrix
aiQuaternion RotationQ;
CalcInterpolatedRotation(RotationQ, AnimationTime, pNodeAnim);
Matrix4f RotationM = Matrix4f(RotationQ.GetMatrix());

// Interpolate translation and generate translation transformation matrix
aiVector3D Translation;
CalcInterpolatedPosition(Translation, AnimationTime, pNodeAnim);
Matrix4f TranslationM;
TranslationM.InitTranslationTransform(Translation.x, Translation.y, Translation.z);

// Combine the above transformations
NodeTransformation = TranslationM * RotationM * ScalingM;
}

Matrix4f GlobalTransformation = ParentTransform * NodeTransformation;

if (m_BoneMapping.find(NodeName) != m_BoneMapping.end()) {
uint BoneIndex = m_BoneMapping[NodeName];
m_BoneInfo[BoneIndex].FinalTransformation = m_GlobalInverseTransform * GlobalTransformation *
m_BoneInfo[BoneIndex].BoneOffset;
}

for (uint i = 0 ; i < pNode->mNumChildren ; i++) {
}
}


What I am not doing at the moment is calling the CalcInterpolatedRotation/Scaling/Position()-functions for testing purposes. Other than that I just converted this tutorial into my own class.

I attached a front view of the thumb. It's not even close to where it should be.

EDIT: I did a bit of further digging into the ASSIMP documentation to find out what some of the matrices represent:

_globalInverseMatrix is the inverted matrix of the root node; Is this only to make sure the model is rotated correctly?

node->mTransformation is the matrix relative to the parent node

this->_bones[boneID]->offsetMatrix is a matrix which transforms from mesh into bone space

#### Attached Thumbnails

Edited by IceBreaker23, 06 March 2014 - 03:08 PM.

### #8Buckeye  GDNet+

Posted 06 March 2014 - 03:12 PM

What's rot_in_quaternion(0.176653 -0.062347 -0.335345)) ? What about the scale?

It appears you're going to have to keep checking values. Is what's coming into your Update function correct or not? If it's not, trace backwards. If it's correct, check the next piece of code that uses it. Somewhere the values will have to be incorrect.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

### #9IceBreaker23  Members

Posted 07 March 2014 - 04:41 AM

After a long debugging session I've come up with the solution.

I downloaded the source code of the tutorial, ran it alongside my application and compared values.

The code is just not very well optimized. It works but I am a bit worried about these many transpose() calls.

Maybe you can come up with a better solution?

void SkinnedMesh::UpdateNode(aiAnimation *animation,float animationTime, aiNode *node, Matrix parentTransform)
{
const char *nodeName = node->mName.C_Str();
const aiNodeAnim *nodeAnim = FindNodeAnim(animation, nodeName);

Matrix NodeTransformation;
TransformMatrix(NodeTransformation,node->mTransformation);

if(nodeAnim != NULL)
{
aiVector3D Scaling;
CalcInterpolatedScaling(Scaling, animationTime, nodeAnim);
Matrix ScalingM = Matrix::CreateScale(Scaling.x,Scaling.y,Scaling.z);
ScalingM = ScalingM.Transpose();

aiQuaternion q;
CalcInterpolatedRotation(q, animationTime, nodeAnim);
Matrix RotationM = Matrix::CreateFromQuaternion(Quaternion(q.x,q.y,q.z,q.w));
RotationM = RotationM.Transpose();

aiVector3D Translation;
CalcInterpolatedPosition(Translation, animationTime, nodeAnim);
Matrix TranslationM = Matrix::CreateTranslation(Translation.x,Translation.y,Translation.z);
TranslationM = TranslationM.Transpose();

NodeTransformation = TranslationM * RotationM * ScalingM;
}

Matrix finalTransform = parentTransform * NodeTransformation;

//+++optimization
if(_boneMap.find(nodeName) != _boneMap.end())
{
int boneID = _boneMap[nodeName];
_finalBoneTransformations[boneID] = (_globalInverseMatrix * finalTransform *  this->_bones[boneID].offsetMatrix).Transpose();
}

for(unsigned int i = 0;i<node->mNumChildren;i++)
{
UpdateNode(animation,animationTime,node->mChildren[i],finalTransform);
}

}


Thanks for your help so far and right now I am so happy that it finally works Up,up and ahead with my engine!

### #10Buckeye  GDNet+

Posted 07 March 2014 - 06:19 AM

Congratulations! Good job.

All the transpose calls would seem to be because the math library you're using produces row-major matrices. The code is written to use column-major matrices. Your shader, into which you stuff the final transforms, expects row-major matrices.

So the sequence is: calculate row-major scale, rot and trans mats. Transpose each one. Multiply them in right-to-left order (an indication of column-major matrices) to get the node transformation. Multiply the node-transform with the parent (right-to-left order because the parent is column-major also). Multiply the result by globalinverse and bone-offsets (column-major) to get the final transform. Transpose the final to make it compatible with shader row-major requirement.

With regard to the name-searching routine to find the right slot to store the final transforms: that's a pretty common "feature" for frame hierarchy and animation controller routines. Many (most? all?) skinned mesh export files specify bones (frames, nodes, etc.) with ASCII name strings. Animation data is really an entirely separate set of information from the bone structure in pose position. However, the animation data must store the final matrices in proper order so, in the shader routine, the bone indices from the frame hierarchy select the correct matrices calculated for animation for proper vertex bone-weighting. Those bone indices are from the hierarchy, not the animation.

With regard to "optimization:" Your code is working, so I wouldn't recommend messing with it. In general, you shouldn't "optimize" code until you've done extensive bench-marking and have determined that a particular section of code is a major contributor to the cycle time AND your application cycle time MUST be reduced AND you know just how much the cycle time MUST be improved AND you've determined that the code can be optimized sufficiently to achieve the desired results.

You COULD change your entire skinned mesh code to use row-major matrices, reverse the order of every matrix multiplication you do, and save a couple of transpose calls. However, a transpose is very fast as it just swaps a few of the matrix values. You would never notice the difference and probably cause your code to be F--- Up Beyond All Recognition (FUBAR). { EDIT: beyond all belief?? no, beyond all "Recognition" }

The name-search is a necessary evil and provides a check that the animation data applies to the frame hierarchy. As mentioned above, animation data is pretty much a separate set of data from your node hierarchy. You can store several sets of animation data (key-frames) in separate files, load them into an animation controller, switch among them, blend them, etc. However, if an animation set (an array of key-frames), perhaps created for a different skinned mesh, doesn't match the node hierarchy, bone-name for bone-name, it's going to be garbage.

In maybe a couple of weeks, I'll be finished with an article on an animation controller that you might find informative. It should feature animation tracks so you can blend two or more animations together.

Edited by Buckeye, 07 March 2014 - 10:27 AM.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

### #11IceBreaker23  Members

Posted 07 March 2014 - 08:38 AM

You are right that optimizing it right now is definitly not a good thing to do.

I will implement animation blending as the next feature in my skinned mesh class and will also abstract the mesh from the skeleton and bones by creating seperate structs/classes for them.

I already have a fairly good definition of how I want my animation system to work but maybe your article will help me

### #12Buckeye  GDNet+

Posted 07 March 2014 - 10:41 AM

I will implement animation blending as the next feature in my skinned mesh class

Just a suggestion, depending on how you implement your skinned mesh class, you may want to consider making an animation controller as a completely separate class. That is, if your skinned mesh class is closely tied with file importing (which it shouldn't be, but that's another story), a separate animation controller can be used with any appropriate skinned mesh. All the animation controller cares about is calculating animation stuff and storing the results somewhere. Rather than having the animation controller get tied in with bone transforms and offsets, etc., just have the animation controller calculate all the animation transforms and store it according to a provided bonename-versus-index array or other simple interface. I have one animation controller that takes an array of pairs <bone-name, pointer>. After it calculates an anim transform, it looks up the key-frame-name in the array and stores the transform in the address provided (which happens to be with the node with the same name in the hierarchy.) That array is static, needs to be filled only once, and lasts the lifetime of the controller. Then the skinned mesh class, which knows all about what's needed to get the shader setup, takes the anim transforms (which magically appear in the correct place after the animation controller is told to "advance the time") and handles the remainder of the process.

That approach separates the skinned mesh from having to deal with multiple animation sets, which sets are active, how they're being mixed, what time that sequence is in which animation, etc. On the other hand, the animation controller doesn't need to deal with transposing or finding nodes in the hierarchy, or anything about offsets, etc. I.e., keep all the pose position info in one place, and all the animated position stuff in another, and relate them only though the node-name.

Edited by Buckeye, 07 March 2014 - 10:57 AM.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

### #13kalle_h  Members

Posted 07 March 2014 - 03:40 PM

Even thought you usually don't want to optimize before its needed you always should clean and simplify all code before starting to add more features. This step is usually most beneficial for actual learning because you need really understand something to make it pretty and simple.

There is fast and simple technique to calculate global skeleton without recursion. http://molecularmusings.wordpress.com/2013/02/22/adventures-in-data-oriented-design-part-2-hierarchical-data/

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.