How To Apply Skinning

Graphics and GPU Programming Programming DX11

Started by Mrfence97 April 08, 2014 04:05 PM

10 comments, last by Buckeye 10 years ago

122

Author

April 08, 2014 04:05 PM

I'm attempting to add animation to my game using skinning; however, I do not know how to actually apply it to my model's vertices.

Currently, I have written my own exporter for Blender that (correctly) outputs the vertex weights for each vertex, and I have loaded that into my DX11 app. But how do I then apply the various bone transforms to the vertices? What information do I need? I intend to again write my own exporter, but looking at the .x format, it appears you export the position, rotation and scale of each bone relative to its parent per frame (which I imagine you then turn into a matrix, for that frame).

But I then go on to read about how I have to transform each vertex into the "Bone space", then "Armature space" (what I imagine the .x file exports), and then finally "World space" (using your world matrix). How do I calculate this "Bone space" matrix? I've tried using the inverse matrix of the bone in it's "rest position" and then multiplying this by the pose matrix for the bone, but this doesn't appear to work.

How do I go about this skinning?

Many thanks for any help you can provide.

Buckeye

10,754

April 08, 2014 04:30 PM

A couple of articles may give you a start on the concept and the process - one on skinned meshes, the other on an animation controller for a skinned mesh. Although the discussion in those articles is based on some DX9 work, I think you'll find the principles applicable.

In particular, the skinned mesh article discusses why and how the vertices are transformed into bone space, and the animation transforms applied to get the vertices back into world space in animated position.

With regard to exporting animations, you needn't export every frame if the rotation motion between key frames is consistent with SLERP (Spherical Linear intERPolation). I think that's what Blender uses between keyframes, in any case.

Tips: ensure the blend weights for any vertex sum to 1, as you can then use more efficient blending shaders. Also, things will work out better in the long run if you keep the bone influences per-vertex to 4 or less. That allows packing blend indices and weights into the vertex format more easily.

If you search for Frank Luna's site (EDIT: it's d3dcoder.net), he's got some D3D11 examples** for download. You'll have to search through it a bit to find the skinning example but the effect file (something-or-other.fx) will give you an idea how a skinned animation shader works.

EDIT: "...exporter for Blender that (correctly) outputs the vertex weights for each vertex.."

And the indices?

**In d3d11CodeSet3, chapter 25, Skinned Mesh - take a look at SsaoNormalDepth.fx. Note: his files, etc., are (C) 2011, Frank Luna, All Rights Reserved. FYI, I have found his books invaluable for learning the ins and outs. I've had trouble get his D3D11 samples compiled, but it's not difficult to rework them.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

Mrfence97

122

Author

April 08, 2014 05:37 PM

Thanks for the links, I shall definitely be having a look at them!

In answer to your question about whether I export the indices correctly, my exporter is based on the OBJ exporter but with vertex weights added, so everything else is exported/loaded fine.

Thanks for your time!

Buckeye

10,754

April 08, 2014 08:48 PM

Thanks for the links, I shall definitely be having a look at them!

In answer to your question about whether I export the indices correctly, my exporter is based on the OBJ exporter but with vertex weights added, so everything else is exported/loaded fine.

Thanks for your time!

Just to be clear, I meant bone indices, not vertex indices. Your vertex format will eventually contain both the bone indices and weights. Just checkin'.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

Samurai Jack

455

April 09, 2014 05:52 AM

To perform skinning you have to do the following steps:

1.) Construct hierarchy (BIND POSE)

2.) Compute local matrices for each joint (BIND POSE)

3.) Compute world matrices for each joint (BIND POSE)

4.) Compute inverse world matrices for each joint (BIND POSE)

5.) Transform all vertices and normals via inverse world matrices

6.) Compute animation local matrices

7.) Compute animation world matrices

8.) Transform all inverse vertices and normals (from point 5.) over animation world matrices (point 7)

Buckeye

10,754

April 09, 2014 01:26 PM

To perform skinning you have to do the following steps:

1.) Construct hierarchy (BIND POSE)

2.) Compute local matrices for each joint (BIND POSE)

3.) Compute world matrices for each joint (BIND POSE)

4.) Compute inverse world matrices for each joint (BIND POSE)

5.) Transform all vertices and normals via inverse world matrices

6.) Compute animation local matrices

7.) Compute animation world matrices

8.) Transform all inverse vertices and normals (from point 5.) over animation world matrices (point 7)

As a very general description of the process, that's correct. However, for the sake of clarity, for a skinned mesh with multiple influence bones per-vertex, steps 5 and 8 are done at frame-render time and are done per-bone with the product of the bone's offset matrix (inverse bind pose) and the bone's animation matrix, followed by multiplication by the bone's weighting factor, and the results of those weighted vertex positions are summed to yield the vertex animated world position.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

Mrfence97

122

Author

April 11, 2014 09:42 PM

To update, I have managed to display the model in bind position correctly (at last!), but when I try to apply any actual animation, it deforms with the correct weights/indices etc. but the model is simply not in the right final position.

My code for generating the "finalMatrix" that is sent to the GPU to display the model in bind position is as follows (based upon the links Buckeye gave):


void CalcBindFinalMatrix(Bone* bone, Matrix &parentMatrix)
{
        bone->combinedMatrix = bone->localMatrix * parentMatrix;
	bone->finalMatrix = bone->offsetMatrix * bone->combinedMatrix;

	for(auto child = bone->children.begin(); child != bone->children.end(); ++child)
	{
		CalcBindFinalMatrix(*child, bone->combinedMatrix);
	}
}

This displays correctly, as expected. However, this function (which is run for every frame of animation) which is almost identical, gives weird transformations, almost like the whole model should be rotated 90 degrees.

The bone space transformations for each frame are originally stored in bone->keyFrames[frameNo], and once multiplied etc. they are also the matrices sent to GPU.


void CalcPoseFinalMatrix(Bone* bone, Matrix &parentMatrix, UINT frameNo)
{
	Matrix com = bone->keyFrames[frameNo] * parentMatrix;
	bone->keyFrames[frameNo] = bone->offsetMatrix * com;

	for(auto child = bone->children.begin(); child != bone->children.end(); ++child)
	{
		CalcPoseFinalMatrix(*child, com, frameNo);
	}
}

Also, by HLSL code for performing the skinning is as follows :


//b = bone indices
//w = bone weights

float4 Pos1 = mul(Pos, xBones[b[0]]);
float4 Pos2 = mul(Pos, xBones[b[1]]);
float4 Pos3 = mul(Pos, xBones[b[2]]);
float4 Pos4 = mul(Pos, xBones[b[3]]);

if(w[0] != 0.0)
{
	Pos = (Pos1 * w[0]) + (Pos2 *  w[1]) + (Pos3 * w[2]) + (Pos4 * w[3]);
}

outputPos = mulWorldViewProj(Pos, xWorld, xView, xProj);

Is there anything inherently wrong with my code to cause these issues? I'll post some pictures of what's going wrong if needed!

Many thanks.

Buckeye

10,754

April 11, 2014 10:06 PM

this function (which is run for every frame of animation) which is almost identical, gives weird transformations, almost like the whole model should be rotated 90 degrees.

How do you call the two functions? That is, you start the iteration with something like CalcBindFinalMatrix( root, identityMatrix ) ? What do those calls look like?

Also, are keyframe matrices calculated for every bone? Sometimes the animation set may not include (for instance) the root frame or other bones that aren't animated. If that's the case, sometimes the assumption is that for any bone which does not appear in the animation set, the bind pose matrix will be used. You can test that by initializing all the keyFrame matrices with the bind pose matrices.

With regard to the shader code, what's the intent of "if (w[0] != 0.0 )"? That looks kind of suspicious. If w[0] is 0, what's Pos supposed to be?

Also, do all vertices have 4 influence bones? If they do not, have you checked that the unused weights and indices are 0?

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

Mrfence97

122

Author

April 11, 2014 10:25 PM

Yes, I call both functions exactly as you've written - the "Bind" one immediately after I have loaded in the localMatrix, and the other one after I have loaded in the animation data.

I'm pretty sure there is a keyframe matrix for each bone, as I said, I've written the exporter myself, and for debugging it exports every frame with every bone, exporting it's position, rotation and scale (all values are exactly the same as a .x file I have also exported as a comparison, so the exporter isn't at fault). I will initialize the key frame matrices with the bind pose matrix, however, just to make sure!

Well in my shader Pos is initially set to the vertex position, so if the weight is zero, I still want it to draw the vertex in it's regular position (again, this is for debugging purposes).

All unused weights are 0 I'm afraid!

Buckeye

10,754

April 12, 2014 12:34 AM

Does the mesh have a parent frame? If so, do you take that frame's transformation matrix into account?

Does either the scene frame or the root frame has a (non-identity) transformation matrix? If so, and you call the Calc functions with root bone and an identity matrix, how do you take that transformation matrix into account?

When you render the bind pose (which you say appears correctly), do you use the same shader calls, setting the bone matrix array from the final matrix array?

All unused weights are 0 I'm afraid!

That's good, and as it should be.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

How To Apply Skinning

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

How To Apply Skinning

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines