Skip to main content
GameDev.net gamedev.net
🔒 Locked

Bone animation implementation problem

Started by BlackJoker Feb 13, 2015 at 9:58 AM 16 replies 5k views
Original Post
BlackJoker
BlackJoker

Hi. I want to implement bone animation in my engine and I faced with the issue that when I multiply all bone matrices and set them to shader, my model is disassembling.

As far as I understood the theory correct, I need multiply bind shape matrix on inverse bind matrix for each bone * key frame matrix for current bone and then go throught the bones hierarhy and multiply each chld bone matrix on parent bone matrix. Then I need to pass the result as matrix array to shader and multiply each position with corresponding final joint matrix * joint weight and then do standart multiplication on World, View, Projection amtrices. That shoud be all, but my implementation is not eorking although I double checked my matrices and all other data. I think problem is in matrices, but I cannot find where exactly.

Here is my method for matrix calculation. Could someone tell me what I am doing wrong here? Maybe I am missing something.


private Matrix[] JointMatrices;
private int counter = 0;
private void CalculateInterprolation(AnimationControllerComponent controller, AnimationComponent animation, Dictionary<ulong, Entity> entities)
      {
         JointMatrices = new Matrix[controller.JointDictionary.Count];
         Dictionary<string, Matrix> final = new Dictionary<string, Matrix>();

         foreach (var inverseBindMatrixName in controller.JointDictionary)
         {
            //multiply bind shape * inverse bind * key frame matrix
            Matrix finalMatrix = controller.BindShapeMatrix * inverseBindMatrixName.Value * animation.Animations[inverseBindMatrixName.Key].Matrices[counter];
            final.Add(inverseBindMatrixName.Key, finalMatrix);
            
         }

         if (counter > animation.Animations.Count - 1)
         {
            counter = 0;
         }

         Stack<Joint> joints = new Stack<Joint>();
         joints.Push(animation.SkeletonList[0]);
         while (joints.Count > 0)
         {
            Joint currentJoint = joints.Pop();

            if (final.ContainsKey(currentJoint.JointName))
            {
               if (currentJoint.Parent != null)
               {
                  //multiply child * parent joint matrix
                  final[currentJoint.JointName] = final[currentJoint.JointName]* final[currentJoint.Parent.JointName];
               }
            }

            foreach (var joint in currentJoint.Children)
            {
               joints.Push(joint);
            }
         }

         int index = 0;
         //copy final matrices to matrix array for setting it to shader
         foreach (var matrix in final)
         {
            JointMatrices[index] = matrix.Value;
            index++;
         }
       

         var renderSystem = (RenderSystem)EntityWorld.SystemManager.GetSystems<RenderSystem>()[0];
         var matrixArray = renderSystem.customEffect.GetVariableByName("jointsArray").AsMatrix();
         //set final matrix array to shader
         matrixArray.SetMatrix(JointMatrices);

      }

Shader code:


matrix jointsArray[10];
...
float4 pos = float4(0,0,0,0);
   // Change the position vector to be 4 units for proper matrix calculations.
   input.position.w = 1.0f;

   //Bone1
   pos += mul(jointsArray[input.jointIndices.x], input.position)*input.jointWeights.x;
   
   //Bone2
   pos += mul(jointsArray[input.jointIndices.y], input.position)*input.jointWeights.y;
   
   //Bone3
   pos += mul(jointsArray[input.jointIndices.z], input.position)*input.jointWeights.z;
   
   //Bone4
   pos += mul(jointsArray[input.jointIndices.w], input.position)*input.jointWeights.w;
   

   // Calculate the position of the vertex against the world, view, and projection matrices.
   output.position = pos;
   
   output.position = mul(output.position, worldMatrix);
   output.position = mul(output.position, viewMatrix);
   output.position = mul(output.position, projectionMatrix);

Before calling that method I have model on image 1 and after - on image 2.

P.S. I am using managed DX 11 with sharpdx .

L. Spiro
L. Spiro
I just crawled out of bed and am in no condition to give accurate advice. Since I already posted and you already saw it but I already edited it to be requested to be deleted, I will replace my request for it to be deleted with at least minimal advice: Pass a single world-view-projection matrix instead of multiplying by 3 matrices in the shader for every vertex.


L. Spiro
I restore Nintendo 64 video-game OST’s into HD! https://www.youtube.com/channel/UCCtX_wedtZ5BoyQBXEhnVZw/playlists?view=1&sort=lad&flow=grid
BlackJoker
BlackJoker

Unfortunately this didnt help. Model still disaaaembling, but I just tried with model with just 1 bone - it works fine. So, I think that my problem is in bone hierarchy. Does multiply parent/child in correct order?

L. Spiro
L. Spiro

Unfortunately this didnt help.

Definitely not. I was still in Dream Land.

Try replacing:
output.position = pos;
with:
output.position = float4( pos.xyz, 1.0 );


L. Spiro


[EDIT]
I’m still being retarded.
The shader should be fine.
I’m not awake enough for the inspection required to give you any worthwhile advice. Fairly sure it is on the CPU side, so see below.
[/EDIT]
I restore Nintendo 64 video-game OST’s into HD! https://www.youtube.com/channel/UCCtX_wedtZ5BoyQBXEhnVZw/playlists?view=1&sort=lad&flow=grid
Buckeye
Buckeye

As far as I understood the theory correct, I need multiply bind shape matrix on inverse bind matrix for each bone * key frame matrix for current bone and then go throught the bones hierarhy and multiply each chld bone matrix on parent bone matrix.

Unfortunately, there are as many names for the various animation matrices as there are programmers, so it's not clear what each of your matrix names represents with respect to a child bone, the child's relationship to the parent bone, or the entire model. I.e., the meanings of "bind shape matrix," "bind matrix," "child bone matrix," and "parent bone matrix" are very important with respect to the order of multiplication.

However, in general, your description doesn't sound correct. You may want to review this article to see how your concept of the theory lines up.

I just tried with model with just 1 bone - it works fine.

That's a good start. I would suggest you now try a model with 2 bones, with a very simple animation -> NO movement. I.e., the animation matrix ("key frame" matrix??) should be the identity matrix. Get that working correctly. Then, and only then, add a very small rotation of one of the two bones and get that working correctly.

EDIT: I agree with L.Spiro - the shader looks correct. If so, then, on the CPU side, you have to ensure both the code and the data are correct. So, start with data you know is correct - the simplest possible model you can come up with beyond the single bone example that works - i.e., two bones, a minimum of vertices, either ONE or NO bone influences per vertex, and NO movement/animation. That is, your jointsArray matrices in the shader should be all identity matrices. It's a trivial case, but you have to start from a known good configuration.

Then animate ONE bone just a very small amount from the bind pose, and continuing continue the investigation.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly. You don't forget how to play when you grow old; you grow old when you forget how to play.
BlackJoker
BlackJoker

I am working with collada files and for now I found out that bone moving in correct axis, but it rotates around world origin instead of joints origin.

So, I multiply bindshape * inverse bind matrix * key frame matrix. Where I am missing?

Buckeye
Buckeye

I multiply bindshape * inverse bind matrix * key frame matrix. Where I am missing?

there are as many names for the various animation matrices as there are programmers, so it's not clear what each of your matrix names represents

You may want to review this article to see how your concept of the theory lines up.

it rotates around world origin instead of joints origin.

I have no idea what "bindshape" means, or what the "bind matrix" is, but the rotation about the origin you describe seems to indicate only applying the key frame matrix is actually being applied.

What you are missing, apparently, is reading the responses you've gotten. One more try: take a look at this article, particularly the section "A Slight Diversion from the Initialization Process," and the section that follows: "The Offset Matrix." It describes exactly the problem you're having.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly. You don't forget how to play when you grow old; you grow old when you forget how to play.
BlackJoker
BlackJoker

I have read that article. Because of diffrerences in terms of terminology, I didnt understand anything.

Who knows the Collada animation terminology, please, point me in correct direction. What I am missing in my code to have correct joint rotation?

For now if there is no rotation in bones, models moving correct, but with rotation - it rotates around the worlds center, but not the joint origin. So, if someone could say directly whats wrong with my code, I will be very appreciate because for now I am getting crazy with that unworking stuff.

L. Spiro
L. Spiro
Khronos™ Group is obnoxious to point of being annoying in their attempt to redefine common terms, such as “pixel” (which they try to call a “fragment”) and “world matrix” (which they try to call a “model matrix”).

It’s hard to know if they are renaming common terms or making up new terms. Originally I suspected “bind shape” was their own annoying term for “bind pose” but it appears to just be the mesh’s regular transform prior to skinning.
https://collada.org/mediawiki/index.php/Skinning

Meaning this is how the object was transformed when a skin was bound to it.
Meaning this is just a normal translation you do on the object prior to skinning. Only COLLADA® supports this, and it is not clear if it is really ever necessary, except unless forced so by COLLADA® itself (normally an object has a standard world transform and is assumed after that to be in the correct space for binding).


Bind matrix is probably the inverse translations of the joints at bind time. The inverse bind pose. Notice that below that it lists “pCylinderShape1-skin-bind_poses”.
In other words, it is likely your “offset matrix”.


L. Spiro
I restore Nintendo 64 video-game OST’s into HD! https://www.youtube.com/channel/UCCtX_wedtZ5BoyQBXEhnVZw/playlists?view=1&sort=lad&flow=grid
BlackJoker
BlackJoker

So, as I can see I multiply offset matrix * animation (key frame) matrix. I have no hierarchy for now. This should do the trick, but it has not rotating correctly at all, but moving in correct direction at the same time. Issue only in rotation around joint origin, which has to be solved by offset (inverse bind) matrix, but as I see it doesnt do that.

I double checked the data and they definately correct.

And I dont know what to do to solve this issue...

Buckeye
Buckeye

In your shader, in the mul function calls, you mix the use of column and row vectors, which is unusual. I.e.,


pos += mul(jointsArray[input.jointIndices.x], input.position)*input.jointWeights.x; // input.position treated as column vector
...
output.position = mul(output.position, worldMatrix); // output.position treated as row vector


I'm not familiar with SharpDx, so I'm not sure whether this code does anything with regard to transposing matrices or not:


var matrixArray = renderSystem.customEffect.GetVariableByName("jointsArray").AsMatrix();
//set final matrix array to shader
matrixArray.SetMatrix(JointMatrices);


Can you show the code for binding the world, view and projection matrices to the shader?

The reason I ask - if I'm not mistaken, you must use mul( vector, matrix ) with transposed matrices, and mul( matrix, vector ) for untransposed matrices. If I'm correct in my assumption, and you do not transpose the world/view/projection matrices before binding them to the shader, that may be causing problems. EDIT: That's based on the assumption that you're defaulting to row-major matrices on the CPU side.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly. You don't forget how to play when you grow old; you grow old when you forget how to play.
BlackJoker
BlackJoker

Here is my code for setting World, View, Projection matrices:


material.Effect.GetVariableByName("viewMatrix")
                           .AsMatrix()
                           .SetMatrix(WorkSpace.CameraController[renderContext].ActiveCamera.ViewMatrix);
                        material.Effect.GetVariableByName("projectionMatrix")
                           .AsMatrix()
                           .SetMatrix(WorkSpace.CameraController[renderContext].ActiveCamera.ProjectionMatrix);
                        material.Effect.GetVariableByName("worldMatrix")
                           .AsMatrix()
                           .SetMatrix(transform.FinalTransformation);
ericrrichards22
ericrrichards22

Are you using the Matrix class factory functions to create your view and projection matrices (PerspectiveLH,LookAtLH, and similar) or are you setting them up manually?

The way that you are multiplying the vertex positions by the bone offset matrices in your shader looks off to me, but I may be wrong. It would depend on whether the model and model loading code that you are loading from uses column or row-major matrices. If I recall, the D3DXMatrix stuff that SharpDX wraps uses row-major matrices,so I have usually settled on forcing all matrices that I use into that form for convenience, so you would want to multiply them like mul( vector, matrix) in the HLSL, rather than mul(matrix, vector) as you have.

For instance, when I used Assimp to load models, I needed to transpose all the matrices that the AssimpNet library loaded, as Assimp uses column-major matrices, in order to make the models render correctly. It's possible you have some sort of format mixup like that somewhere in your pipeline that is causing problems.

Shameless plug: Here is a project I got working using SlimDX and bone animations, it might be helpful to compare with your code.

BlackJoker
BlackJoker

Lets clarify first does I need to transpose all joint/animation matrices during importing them from Collada to DirectX or I should use them as is?

Update: I transposed matrices and change multiplication order in shader, but result the same - joints rotating around world center.

Buckeye
Buckeye

Lets clarify first does I need to transpose all joint/animation matrices during importing them from Collada to DirectX or I should use them as is?

Update: I transposed matrices and change multiplication order in shader, but result the same - joints rotating around world center.

To answer your question directly: you need to correctly manipulate Collada data for use with SharpDX. That isn't directly related to D3D11 or SharpDX, other than you're using those APIs to import and render the data.

N.B., for a program to work correctly, the code must be correct, and the data fed to the code must be correct.

You have chosen to create an application based on the SharpDX API, which wraps the DirectX API. Your program imports Collada data and arranges that data in a hierarchical structure. Your program creates an effect which uses a shader written in HLSL intended to apply the principles of skinned mesh (skeletal) animation using matrices. As mentioned, all that code, and all that data must be correct, to get the desired result.

The possibility does exist that someone here on gamedev may "fix" your program for you. You may find a problem by hacking - i.e., changing something here or there that might be the problem. Either way, that may be a difficult and time-consuming process.

However, at this point in the discussion, it appears you may problems both with your program's code, and with the data you're importing. You will likely get help more quickly if you're willing to help others help you by determining yourself what short sections of your code or what particular data do not appear to be correct. You can do that by taking a structured approach to debugging, and by learning debugging techniques. It appears that you've taken a first step by using a very simple model. Continue the process, step-by-step, in a structured manner.

The idea is to change the direction of your future posts toward: "I've verified the values going into these few lines of code are correct. I want the next few lines to [do something you describe]. However, the resulting values are [specific data you describe or post]. How can I better determine the problem at this point?" That will be as opposed to "Here's several sections of code. The model is disassembled. What's wrong?" happy.png

If you choose to do some structured debugging, I would suggest you start by determining that you import Collada data correctly. As has been mentioned above, Collada uses its own terminology, and possibly a coordinate system that needs to be taken into consideration, and the data must be manipulated correctly for use with your program. Note: that will require that you, at a minimum, have a familiarity with the Collada format, know what data you need for your program, and recognize (by actual examination of the data in your program) that you've imported that data correctly.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly. You don't forget how to play when you grow old; you grow old when you forget how to play.
Aardvajk
Aardvajk
Well said, Buckeye, excellent advice.

To the OP, I had a lot of trouble getting this stuff working myself. I had to strip it down to a single bone and a few vertices, and if I recall I had to figure it out CPU style before I could get the shader approach working.

If you work with a minimal data set and do the transforms in software rather than in the GPU, it is MUCH easier to debug. Once it is working there, you have a working example to transfer to shaders.
kbaird
kbaird

I've done collada skinned stuff for xna and directx, and both times I had a lot of coordinate system problems. You appear to have the base mesh inside out stuff figured out, and looking at my code it appears there are no tricks with rotation keys that I can see. Oddly I don't do any swapping or negating on translation keys for DirectX.

I do a lot of strange stuff to the inverse bind pose matrices. Loading them from the collada float array I load them in M11, M21, M31, M41, M12, M22, etc... which might be a sort of transpose as you load. Then I invert, do the below right hand to left hand conversion that I think I got from this very forum, then multiply by a rotation matrix of pi/2 in X, then invert back. Sounds odd but it works from maxland to directxland.


public static void RightHandToLeft(ref Matrix mat)
{
	mat.M31	=-mat.M31;
	mat.M32	=-mat.M32;
	mat.M33	=-mat.M33;
	mat.M34	=-mat.M34;

	mat.M13	=-mat.M13;
	mat.M23	=-mat.M23;
	mat.M33	=-mat.M33;
	mat.M43	=-mat.M43;
}

I do two things that make my implementation not really collada compliant. I assert that the up axis of the file is Z, and the bind shape matrix is identity. I do that because I can use a single set of inverse bind poses for a character, including all of the outfits and extra parts. This means I'll have one set of bones to feed to the shader and I can compute bone bounds and animate and such once.

Looking at your code, it looks like you do the inverseBind multiply over all bones, then do a parent multiply below, but I can't really follow what that is going to do to the bones in my noggin. I have a recursive routine that multiplies by parents on the way out, then multiplies the inverse bind pose by the retrieved matrix like matrix = inverseBindPose * matrix. Order matters.

I think I do this because I sort of do animation in the coordinate system of whatever exported it, in my case Max. That inverse bind pose multiply sort of gets things back into a directx friendly coordinate system because of the aforementioned right to left conversion.

The resulting set of bones is what I pass into the shader.

Shader side, I compute a skin transform and do the position multiply once, but I'm not sure it matters for correctness. You'll need it for the normals anyway.


//ints for > sm2
float4x4 GetSkinXForm(int4 bnIdxs, half4 bnWeights, float4x4 bones[MAX_BONES])
{
	float4x4 skinTransform	=bones[bnIdxs.x] * bnWeights.x;

	skinTransform	+=bones[bnIdxs.y] * bnWeights.y;
	skinTransform	+=bones[bnIdxs.z] * bnWeights.z;
	skinTransform	+=bones[bnIdxs.w] * bnWeights.w;
	
	return	skinTransform;
}

//skin pos and normal
VVPosNorm ComputeSkin(VPosNormBone input, float4x4 bones[MAX_BONES])
{
	VVPosNorm	output;
	
	float4	vertPos	=float4(input.Position, 1);
	
	//generate the world-view-proj matrix
	float4x4	wvp	=mul(mul(mWorld, mView), mProjection);
	
	//do the bone influences
	float4x4 skinTransform	=GetSkinXForm(input.Blend0, input.Weight0, bones);
	
	//xform the vert to the character's boney pos
	vertPos	=mul(vertPos, skinTransform);
	
	//transform the input position to the output
	output.Position	=mul(vertPos, wvp);

	//skin transform the normal
	float3	worldNormal	=mul(input.Normal.xyz, skinTransform);
	
	//world transform the normal
	output.Normal	=mul(worldNormal, mWorld);

	return	output;
}

BlackJoker
BlackJoker

OK, digging around I found an issue:

1. I import collada to DirectX, so all matrices should be transposed.

2. I use Yup axis and Blender has Zup axis, so I have to change axis with special matrix

M11 - 1

M23 - 1

M32 - 1

M44 - 1

multiply each joint matrix once like this - newjointMatrix = axisChangeMatrix * jointMatrix * axisChangeMatrix

You need to do this once during load.

Then I should just multiply offset * keyframeMatrix. Or If I have hierarchy - keyframe*keyFrame.Parent *..... Parent and so on.

Then - send final matrices array to shader.

Thats all.

Now animation works correct.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.