ASSIMP skinned mesh with DX9 problem

Graphics and GPU Programming Programming

Started by Florian22222 March 06, 2014 10:14 AM

11 comments, last by kalle_h 10 years, 1 month ago

Florian22222

618

Author

March 06, 2014 10:14 AM

Hello!

With the help of the forum I managed to fix my skinning shader problem. Now I have another problem.

I am using ASSIMP and DirectX9. I implemented the skeletal animation according to this tutorial: http://ogldev.atspace.co.uk/www/tutorial38/tutorial38.html

If I use the default pose(T-pose) the model is shown just how it should be(see attached screenshot model_tpose.png).

As soon as I use the animations from animation frame0 it goes all weird(see attached screenshot model_frame0.png).

I think that something with calculating the NodeTransformation is wrong.

Here is the code of the update function called each frame(the Matrix class is from SimpleMath.h, an oop wrapper for DirectXMath.h):


void SkinnedMesh::UpdateNode(aiAnimation *animation,float animationTime, aiNode *node, Matrix parentTransform)
{
	const char *nodeName = node->mName.C_Str();
	const aiNodeAnim *nodeAnim = FindNodeAnim(animation, nodeName);

	Matrix NodeTransformation(*(Matrix *)&node->mTransformation);

        //if I comment the whole if-section then it works, because the bones are then in default pose
	if(nodeAnim != NULL)
	{
                //somewhere here should be the problem
		aiVector3D Scaling = nodeAnim->mScalingKeys[0].mValue;
		Matrix ScalingM = Matrix::CreateScale(Scaling.x,Scaling.y,Scaling.z);

		aiQuaternion q = nodeAnim->mRotationKeys[0].mValue;
		Matrix RotationM = Matrix::CreateFromQuaternion(Quaternion(q.x,q.y,q.z,q.w));;

		aiVector3D Translation = nodeAnim->mPositionKeys[0].mValue;
		Matrix TranslationM = Matrix::CreateTranslation(Translation.x,Translation.y,Translation.z);

                NodeTransformation = TranslationM * RotationM * ScalingM;
	}

	Matrix finalTransform = parentTransform * NodeTransformation;

	if(_boneMap.find(nodeName) != _boneMap.end())
	{
		int boneID = _boneMap[nodeName];
		_finalBoneTransformations[boneID] = _globalInverseMatrix * finalTransform *  this->_bones[boneID].offsetMatrix;
	}

	for(unsigned int i = 0;i<node->mNumChildren;i++)
	{
		UpdateNode(animation,animationTime,node->mChildren[i],finalTransform);
	}

}

I hope someone can help me and tell me why it does that.

Btw: the problem is likely not right-handed/left-handed matrices problem, since ASSIMP and SimpleMath both use right-handed matrices.

Thanks in advance!

Buckeye

10,754

March 06, 2014 02:21 PM

I think that something with calculating the NodeTransformation is wrong.

If the pose position is fine (in the left image) when you comment out the section you indicate, that seems likely. But see my comment below about the Matrix NodeTransformation line.

I haven't made the transition to DirectXMath yet, but, just on inspection, the form of the calculation in if( nodeAnim != NULL) {...} looks good. Normally I would recommend (as with your shader problem): if the code looks good, check the input/output values. However, that's not quite as easy here because it's a bit difficult to know what the correct values should be. Also, because you're doing calcs for the entire frame hierarchy, there's more than just one value to check.

I'm not familiar with how C# works under the hood, so I'm just trying to understand what everything does. I'm a bit curious about this line:


Matrix NodeTransformation(*(Matrix *)&node->mTransformation);

Does it copy mTransformation or create a reference to it? If it's just a copy, that's fine. My concern is that you may be storing the calculated animation transform back in the node. If you're not sure, you can check by visually examining node->mTransformation before and after the animation calculation. node->mTransformation should not have been modified. I understand that you set up the code to display the pose position for testing, but normally you shouldn't need node->mTransformation to calculate the final transforms.

If you're absolutely positive that NodeTransformation is just a copy, then you might want to try to isolate the problem to a single value. First, the recommendation and then the explanation.

Pick a node in your frame hierarchy very high in the tree - like an arm or a hand. Then use something like the following, where "Left_Hand" is an actual bone name in your structure:


if( nodeAnim != NULL && nodeName.c_str() == "Left_Hand")
{
   ....
}

What that should do is render most of the model in pose position and apply the animation to only the hand and its children. If the pose position parts look good, then the problem must be related to calculating the animation transform, retrieving the animation transform [ FindNodeAnim(animation, nodeName) ], accessing the key values or creating the animated NodeTransformation.

It's a start.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

Florian22222

618

Author

March 06, 2014 03:24 PM

For a start it's not C#

I changed the line


Matrix NodeTransformation(*(Matrix *)&node->mTransformation);


Matrix NodeTransformation = Matrix(*(Matrix *)&node->mTransformation);

because I wasn't sure about this constructor call as well. I checked and the matrix-constructor takes a const value.(so no modification possible)

I limited the if(nodeAnim != NULL) to only work with "thumb.L". The result is attached as a screenshot.

EDIT: the object beside the hand is a lantern.

Buckeye

10,754

March 06, 2014 04:24 PM

It's not C#. Well, there you go. As mentioned, I'm not familiar with C# and proved the point!

EDIT: my first try at this post was total garbage and I deleted it. Back to the drawing board for a moment.

First: from what you've done, it definitely looks like the problem is related to accessing the animation keys, doing calcs with the keys, and using the result for the final transform.

I don't know what some of the functions do.

What does FindNodeAnim(animation, nodeName) do? I.e., what's the nodeAnim object?

In nodeAnim->mScalingKeys[0].mValue, what is the subscript "[0]" ?

Are you accessing the animation timed key-frames directly?

You don't use input animationTime. Are you just doing calcs for time = 0 for testing purposes?

You can check do a bit of a check on actual values. Many animations use only rotations. If that's true in this case, the scale key should be ( 1, 1, 1) and the translate key should be the child's position with respect to the parent and not likely to change. That is, the animation translate key for the thumb should be the same as the pose position translate key. Don't know how you might check that if mTransformation is being reused for animation calcs.

EDIT2:


aiQuaternion q = nodeAnim->mRotationKeys[0].mValue;
Matrix RotationM = Matrix::CreateFromQuaternion(Quaternion(q.x,q.y,q.z,q.w));;

Just looking: why the double ";;" ?

Also, don't know the requirements for the various functions, but can you create the matrix from mValue directly? or from q itself?

EDIT3: Note - this approach is pretty much the same as with your shader problem. Be sure you know what the code should be doing and is coded to do that; check that the values going coming into the calc are correct; check that the resulting values are correct. Somewhere along that line is the problem.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

Florian22222

618

Author

March 06, 2014 05:29 PM

The function FindNodeAnim looks in all the animation's channels to find the respective aiAnimNode* to the bone-name.

Here is the function FindNodeAnim:


aiNodeAnim *SkinnedMesh::FindNodeAnim(aiAnimation *animation,const char *nodeName)
{
	for(unsigned int i = 0;i<animation->mNumChannels;i++)
	{
		if(strcmp(nodeName,animation->mChannels[i]->mNodeName.C_Str()) == 0)
		{
			return animation->mChannels[i];
		}
	}

	return NULL;
}

For testing purposes I only access time = 0(so I dont have the interpolation code in as well, which would make the whole debugging even more complex)

The double ";;" was just a typo and I removed it.

Part of the problem is that there are 2 libraries I need to convert between. One is the ASSIMP-math library(respective data types are aiQuaternion,aiVector3) and the SimpleMath library(http://blogs.msdn.com/b/shawnhar/archive/2013/01/08/simplemath-a-simplified-wrapper-for-directxmath.aspx)

I can also create an aiMatrix4x4 from the aiQuaternion and then cast it to my Matrix-type. I actually tried it and it gives same result.

I tried to isolate the rotation as you suggested. Now the code is:


void SkinnedMesh::UpdateNode(aiAnimation *animation,float animationTime, aiNode *node, Matrix parentTransform)
{
	const char *nodeName = node->mName.C_Str();
	const aiNodeAnim *nodeAnim = FindNodeAnim(animation, nodeName);

	Matrix NodeTransformation = Matrix(*(Matrix *)&node->mTransformation);

	if(nodeAnim != NULL && strcmp(nodeName,"thumb.L") == 0)
	{
		aiQuaternion q = nodeAnim->mRotationKeys[0].mValue;
		Matrix RotationM = Matrix::CreateFromQuaternion(Quaternion(q.x,q.y,q.z,q.w));

		NodeTransformation = Matrix::CreateTranslation(NodeTransformation.m[0][3],NodeTransformation.m[1][3],NodeTransformation.m[2][3]) * RotationM * Matrix::CreateScale(1.0f);
	}

	Matrix finalTransform = parentTransform * NodeTransformation;

	if(_boneMap.find(nodeName) != _boneMap.end())
	{
		int boneID = _boneMap[nodeName];
		_finalBoneTransformations[boneID] = _globalInverseMatrix * finalTransform *  this->_bones[boneID].offsetMatrix;
	}

	for(unsigned int i = 0;i<node->mNumChildren;i++)
	{
		UpdateNode(animation,animationTime,node->mChildren[i],finalTransform);
	}
}

The output is the same as in the previously posted screenshot with the weird left thumb.

EDIT: I know its something similar to my shader problem. The only problem is that I do not know the math behind quaternions and so checking values doesnt help very much.

Buckeye

10,754

March 06, 2014 05:52 PM

Looks like you're in a similar situation as with your shader problem. If the code looks right, you have to check the values.

I don't know if your import file is in text or binary format. If it's in text, find key-frame 0 for the thumb and see if that's what you're getting from the mValues. If the import is binary, that's a bit more difficult. As I noted in one of my edits above, you can at least check if the mValues are reasonable.

For instance, given that the thumb's "to-parent" transform should keep it close to its parent, it looks like it's getting a pretty big translate or scale component. Is that's what coming into the update function, or out from it? You don't show the front view of the "weird" thumb. Is it's position relative to the hand strangely the same as the distance it should be from the root node?

You assume the key-frame data for a single node (thumb, in this case) is a local transform, i.e., with respect to its parent. That's how you use the data. Is that a correct assumption?

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

Florian22222

618

Author

March 06, 2014 06:46 PM

I just checked frame0 in my .md5anim file and compared it. The translation as well as the rotation values that I get from mValue are correct and reasonable(trans(0.054595 0.018457 -0.015135), rot_in_quaternion(0.176653 -0.062347 -0.335345))

I got this model from the tutorial(http://ogldev.atspace.co.uk/www/tutorial38/tutorial38.html) and also just translated part of the code into my own class.

Here is the original update function of the tutorial:


void Mesh::ReadNodeHeirarchy(float AnimationTime, const aiNode* pNode, const Matrix4f& ParentTransform)
{ 
    string NodeName(pNode->mName.data);

    const aiAnimation* pAnimation = m_pScene->mAnimations[0];

    Matrix4f NodeTransformation(pNode->mTransformation);

    const aiNodeAnim* pNodeAnim = FindNodeAnim(pAnimation, NodeName);

    if (pNodeAnim) {
        // Interpolate scaling and generate scaling transformation matrix
        aiVector3D Scaling;
        CalcInterpolatedScaling(Scaling, AnimationTime, pNodeAnim);
        Matrix4f ScalingM;
        ScalingM.InitScaleTransform(Scaling.x, Scaling.y, Scaling.z);

        // Interpolate rotation and generate rotation transformation matrix
        aiQuaternion RotationQ;
        CalcInterpolatedRotation(RotationQ, AnimationTime, pNodeAnim); 
        Matrix4f RotationM = Matrix4f(RotationQ.GetMatrix());

        // Interpolate translation and generate translation transformation matrix
        aiVector3D Translation;
        CalcInterpolatedPosition(Translation, AnimationTime, pNodeAnim);
        Matrix4f TranslationM;
        TranslationM.InitTranslationTransform(Translation.x, Translation.y, Translation.z);

        // Combine the above transformations
        NodeTransformation = TranslationM * RotationM * ScalingM;
    }

    Matrix4f GlobalTransformation = ParentTransform * NodeTransformation;

    if (m_BoneMapping.find(NodeName) != m_BoneMapping.end()) {
        uint BoneIndex = m_BoneMapping[NodeName];
        m_BoneInfo[BoneIndex].FinalTransformation = m_GlobalInverseTransform * GlobalTransformation * 
                                                    m_BoneInfo[BoneIndex].BoneOffset;
    }

    for (uint i = 0 ; i < pNode->mNumChildren ; i++) {
        ReadNodeHeirarchy(AnimationTime, pNode->mChildren[i], GlobalTransformation);
    }
}

What I am not doing at the moment is calling the CalcInterpolatedRotation/Scaling/Position()-functions for testing purposes. Other than that I just converted this tutorial into my own class.

I attached a front view of the thumb. It's not even close to where it should be.

EDIT: I did a bit of further digging into the ASSIMP documentation to find out what some of the matrices represent:

_globalInverseMatrix is the inverted matrix of the root node; Is this only to make sure the model is rotated correctly?

node->mTransformation is the matrix relative to the parent node

this->_bones[boneID]->offsetMatrix is a matrix which transforms from mesh into bone space

Buckeye

10,754

March 06, 2014 09:12 PM

What's rot_in_quaternion(0.176653 -0.062347 -0.335345)) ? What about the scale?

It appears you're going to have to keep checking values. Is what's coming into your Update function correct or not? If it's not, trace backwards. If it's correct, check the next piece of code that uses it. Somewhere the values will have to be incorrect.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

Florian22222

618

Author

March 07, 2014 10:41 AM

After a long debugging session I've come up with the solution.

I downloaded the source code of the tutorial, ran it alongside my application and compared values.

The code is just not very well optimized. It works but I am a bit worried about these many transpose() calls.

Maybe you can come up with a better solution?


void SkinnedMesh::UpdateNode(aiAnimation *animation,float animationTime, aiNode *node, Matrix parentTransform)
{
	const char *nodeName = node->mName.C_Str();
	const aiNodeAnim *nodeAnim = FindNodeAnim(animation, nodeName);

	Matrix NodeTransformation;
	TransformMatrix(NodeTransformation,node->mTransformation);

	if(nodeAnim != NULL)
	{
		aiVector3D Scaling;
		CalcInterpolatedScaling(Scaling, animationTime, nodeAnim);
		Matrix ScalingM = Matrix::CreateScale(Scaling.x,Scaling.y,Scaling.z);
		ScalingM = ScalingM.Transpose();

		aiQuaternion q;
		CalcInterpolatedRotation(q, animationTime, nodeAnim); 
		Matrix RotationM = Matrix::CreateFromQuaternion(Quaternion(q.x,q.y,q.z,q.w));
		RotationM = RotationM.Transpose();

		aiVector3D Translation;
		CalcInterpolatedPosition(Translation, animationTime, nodeAnim);
		Matrix TranslationM = Matrix::CreateTranslation(Translation.x,Translation.y,Translation.z);
		TranslationM = TranslationM.Transpose();

        NodeTransformation = TranslationM * RotationM * ScalingM;
	}

	Matrix finalTransform = parentTransform * NodeTransformation;

	//+++optimization
	if(_boneMap.find(nodeName) != _boneMap.end())
	{
		int boneID = _boneMap[nodeName];
		_finalBoneTransformations[boneID] = (_globalInverseMatrix * finalTransform *  this->_bones[boneID].offsetMatrix).Transpose();
	}

	for(unsigned int i = 0;i<node->mNumChildren;i++)
	{
		UpdateNode(animation,animationTime,node->mChildren[i],finalTransform);
	}

}

Thanks for your help so far and right now I am so happy that it finally works :) Up,up and ahead with my engine!

Buckeye

10,754

March 07, 2014 12:19 PM

Congratulations! Good job.

All the transpose calls would seem to be because the math library you're using produces row-major matrices. The code is written to use column-major matrices. Your shader, into which you stuff the final transforms, expects row-major matrices.

So the sequence is: calculate row-major scale, rot and trans mats. Transpose each one. Multiply them in right-to-left order (an indication of column-major matrices) to get the node transformation. Multiply the node-transform with the parent (right-to-left order because the parent is column-major also). Multiply the result by globalinverse and bone-offsets (column-major) to get the final transform. Transpose the final to make it compatible with shader row-major requirement.

With regard to the name-searching routine to find the right slot to store the final transforms: that's a pretty common "feature" for frame hierarchy and animation controller routines. Many (most? all?) skinned mesh export files specify bones (frames, nodes, etc.) with ASCII name strings. Animation data is really an entirely separate set of information from the bone structure in pose position. However, the animation data must store the final matrices in proper order so, in the shader routine, the bone indices from the frame hierarchy select the correct matrices calculated for animation for proper vertex bone-weighting. Those bone indices are from the hierarchy, not the animation.

With regard to "optimization:" Your code is working, so I wouldn't recommend messing with it. In general, you shouldn't "optimize" code until you've done extensive bench-marking and have determined that a particular section of code is a major contributor to the cycle time AND your application cycle time MUST be reduced AND you know just how much the cycle time MUST be improved AND you've determined that the code can be optimized sufficiently to achieve the desired results.

You COULD change your entire skinned mesh code to use row-major matrices, reverse the order of every matrix multiplication you do, and save a couple of transpose calls. However, a transpose is very fast as it just swaps a few of the matrix values. You would never notice the difference and probably cause your code to be F--- Up Beyond All Recognition (FUBAR). { EDIT: beyond all belief?? no, beyond all "Recognition" }

The name-search is a necessary evil and provides a check that the animation data applies to the frame hierarchy. As mentioned above, animation data is pretty much a separate set of data from your node hierarchy. You can store several sets of animation data (key-frames) in separate files, load them into an animation controller, switch among them, blend them, etc. However, if an animation set (an array of key-frames), perhaps created for a different skinned mesh, doesn't match the node hierarchy, bone-name for bone-name, it's going to be garbage.

In maybe a couple of weeks, I'll be finished with an article on an animation controller that you might find informative. It should feature animation tracks so you can blend two or more animations together.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

ASSIMP skinned mesh with DX9 problem

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

ASSIMP skinned mesh with DX9 problem

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines