Jump to content

  • Log In with Google      Sign In   
  • Create Account

Skinning: problem calculating bindpose/offset-matrix


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
10 replies to this topic

#1 Juliean   GDNet+   -  Reputation: 2755

Like
0Likes
Like

Posted 21 February 2014 - 08:31 AM

Hello,

 

while implementing skinning support to my engine and therefore my mesh file format, I'm down to one problem: The inverse bind pose matrices (at least of my test model) don't match with what would be needed to render it correctly. I've converted the Tiny.x - mesh from the DirectX9-SDK to my own mesh format, and I've verified up until now that all data is 100% matching, minus the bind pose/offset matrix.

 

I am calculating the matrix by traversing the bone hierachy, and then inversing the absolute transform of each bone:

void MeshSkeleton::ProcessSkeleton(void)
{
	m_vMatrices.clear();
	size_t numBone = 0;

	ProcessBone(m_root, nullptr, numBone);
}

void MeshSkeleton::ProcessBone(Bone& bone, Bone* pParent, size_t& numBone)
{
	if(pParent)
		bone.mAbsolute = bone.mRelative * pParent->mAbsolute;
	else
		bone.mAbsolute = bone.mRelative;
			
	// in case the inverse bind poses have already been calculated, output the final bone matrix
	if(!m_vInvBindPoses.empty())
		m_vMatrices.push_back(m_vInvBindPoses[numBone] * bone.mAbsolute);
	else
		m_vMatrices.push_back(math::MatIdentity());

	numBone++;

	for(auto& child : bone.m_vChildren)
	{
		ProcessBone(child, &bone, numBone);
	}
}

void MeshSkeleton::CalculateBindPoses(void)
{
	m_vInvBindPoses.clear();
	CalculateBindPose(m_root);
}

void MeshSkeleton::CalculateBindPose(Bone& bone)
{
	m_vInvBindPoses.push_back(bone.mAbsolute.inverse());

	for(auto& child : bone.m_vChildren)
	{
		CalculateBindPose(child);
	}
}

First, ProcessSceleton is called, then CalculateBindPoses. The absolute matrices are identically to those of the Tiny.x-mesh when loaded in the sample, so there should be no problem here. However, when I look at the offset matrices from the sample:

    // get each of the bone offset matrices so that we don't need to get them later
        for( iBone = 0; iBone < cBones; iBone++ )
 {
            pMeshContainer->pBoneOffsetMatrices[iBone] = *( pMeshContainer->pSkinInfo->GetBoneOffsetMatrix( iBone ) );
        }

They aren't even similiar to mine in any way shape or form (I checked the correct bones).

 

Now I can fix this by storing the offset matrix from pSkinInfo in the mesh file and load them with the file (that way the animation works correctly), but I am wondering about the reasoning behind this. Is there some error in my calculations? Or is the offset-matrix calculated differently in the DirectX9-file format?


Edited by Juliean, 21 February 2014 - 08:32 AM.


Sponsor:

#2 Buckeye   Crossbones+   -  Reputation: 6407

Like
1Likes
Like

Posted 21 February 2014 - 10:04 AM

Need a bit more information. What are the matrices "mRelative" and "mAbsolute?" Are those the bind pose bone matrices "relative-to-parent" and "relative-to-root?" Without being sure, I can't really tell if your code is correct.

 

 


Now I can fix this by storing the offset matrix from pSkinInfo in the mesh file and load them with the file (that way the animation works correctly), but I am wondering about the reasoning behind this.

It's not clear what you're asking. If you're loading tiny.x anyway, why not use the offsets?

 

 

 


is the offset-matrix calculated differently in the DirectX9-file format?

The offset matrix is the inverse of the to-root matrix for each bone. I think the to-root matrix is what you're calling mAbsolute, each bone's relativeMat * parent.to-root, calculated hierarchically.

 

 


// in case the inverse bind poses have already been calculated, output the final bone matrix if(!m_vInvBindPoses.empty()) m_vMatrices.push_back(m_vInvBindPoses[numBone] * bone.mAbsolute); else m_vMatrices.push_back(math::MatIdentity());

Not sure what you're doing here. You have another routine for doing the calc. EDIT: And if the bindposes have been calculated, why are you multiplying them (again) by the absolute?

 

EDIT: The biggest question - are you sure the bone numbers you use (ordered by hierarchy), are in the correct order to match the skininfo numbers? You use only children of frames, so I assume you somewhere resolve sibling frames into another child of the parent. Is that correct? The array of final matrices has to be in skininfo numbered order. I'm not sure that your hierarchy necessarily matches. You can check that using skinfo-GetBoneName(boneNumber) and checking your bonename-vs-bonenumber.

 

[ I wrote an article on animating skinned meshes using matrices, very similar to what you're doing. Unfortunately, it's still in moderator approval. Hopefully it won't be too long before it's available. ]


Edited by Buckeye, 21 February 2014 - 10:30 AM.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.


#3 Juliean   GDNet+   -  Reputation: 2755

Like
0Likes
Like

Posted 21 February 2014 - 10:31 AM


Need a bit more information. What are the matrices "mRelative" and "mAbsolute?" Are those the bind pose bone matrices "relative-to-parent" and "relative-to-root?" Without being sure, I can't really tell if your code is correct.

 

I'll try to explain things in more detail. mRelative is the boens local transform, relative to the parent - at least I assume, since I'm multiplying it with the parents global transform (mAbsolute) iteratively. I'm really not that into the terminology, I hope it makes a little more sense that way. If you look at the Tiny.x-file, the "relative" matrices would be those:

Frame Scene_Root {
 

 FrameTransformMatrix {
  1.000000,0.000000,0.000000,0.000000,0.000000,1.000000,0.000000,0.000000,0.000000,0.000000,1.000000,0.000000,0.000000,0.000000,0.000000,1.000000;;
 }

 Frame body {
  

  FrameTransformMatrix {
   1.278853,0.000000,-0.000000,0.000000,0.000000,0.000000,1.123165,0.000000,0.000000,-1.470235,0.000000,0.000000,0.135977,2.027985,133.967667,1.000000;;
  }	

  Frame {
   

   FrameTransformMatrix {
    1.000000,-0.000000,-0.000000,0.000000,-0.000000,1.000000,0.000000,0.000000,-0.000000,0.000000,1.000000,0.000000,-0.142114,0.000023,-49.556850,1.000000;;
   }

equivalent to the frame-transform matrices here, and "absolute" would be calculated by multiplying them up over the hierachy.

 


It's not clear what you're asking. If you're loading tiny.x anyway, why not use the offsets?

 

I mentioned (though sparsly) in the beginning that I have a custom mesh file that I use for rendering, I'm only loading Tiny.x in my mesh-converter to port it to my format, mainly since I don't have access to a modelling program at the moment and needed something to test. I can store the offsets in my mesh format, sure, but I'd nevertheless like to know why I can't calculate the correct inverse bind pose myself, and whether this is a fault with my algorithm, or the way x-files are set up (or maybe some issue with the tiny-mesh itself?).

 


Not sure what you're doing here. You have another routine for doing the calc. EDIT: And if the bindposes have been calculated, why are you multiplying them (again) by the absolute?

 

I calculate the final matrices that gets sent to the shader for each bone there. This method gets called once in the beginning before the inv-bind poses are calculated, and every time the skeleton is updated, so that the final matrices for the shader are recalculated. This would create all identity matrices without animation, but once I start to apply the animation, I get the matrices that move the vertices. Again, apologies for my inability to use the right terminology, I hope you can still follow me.

 


[ I wrote an article on animating skinned meshes using matrices, very similar to what you're doing. Unfortunately, it's still in moderator approval. Hopefully it won't be too long before it's available. ]

I'm kind of sad that it wasn't available earlier, could have probably saved me a few hours of lifetime. I'll check it out once it is open anyway!



#4 Buckeye   Crossbones+   -  Reputation: 6407

Like
1Likes
Like

Posted 21 February 2014 - 10:37 AM

Be sure to see the edited comments in my last post. The index order of the bone matrices is likely the problem.


Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.


#5 Juliean   GDNet+   -  Reputation: 2755

Like
0Likes
Like

Posted 21 February 2014 - 10:51 AM


Be sure to see the edited comments in my last post. The index order of the bone matrices is likely the problem.

 

Ah, I must have finished my post shortly before your edit ;)

 


EDIT: The biggest question - are you sure the bone numbers you use (ordered by hierarchy), are in the correct order to match the skininfo numbers? You use only children of frames, so I assume you somewhere resolve sibling frames into another child of the parent. Is that correct? The array of final matrices has to be in skininfo numbered order. I'm not sure that your hierarchy necessarily matches. You can check that using skinfo-GetBoneName(boneNumber) and checking your bonename-vs-bonenumber.

 

This was a problem, yes, but I solved it already. It is not directly related to portion of code I showed - the skininfo-bone order appears to only matter for what I supply to the shader. I still have to update the hierachy like I did in my code in frame-order, and for supplying the final matrices to the shader, I sort the matrices by the bone names from the skinInfo. The animation is working nicely that way, using the skininfo-offset-matrices, yet I still wonder what DirectX does differently to me to calculate their offset matrices. Do I maybe need to take the skininfo-order into account instead of the frame-hierachy for calculating the matrix here as well? (I wouldn't know how, though...)



#6 Buckeye   Crossbones+   -  Reputation: 6407

Like
0Likes
Like

Posted 21 February 2014 - 10:55 AM

Again, if you didn't see previous post: Check the order of the matrix array to ensure they are in proper bone index order.




Buckeye, on 21 Feb 2014 - 11:04 AM, said:


Not sure what you're doing here. You have another routine for doing the calc. EDIT: And if the bindposes have been calculated, why are you multiplying them (again) by the absolute?



I calculate the final matrices that gets sent to the shader for each bone there. This method gets called once in the beginning before the inv-bind poses are calculated, and every time the skeleton is updated, so that the final matrices for the shader are recalculated. This would create all identity matrices without animation, but once I start to apply the animation, I get the matrices that move the vertices. Again, apologies for my inability to use the right terminology, I hope you can still follow me.


 

 

I think I'm getting a better idea of what you're intent is. Check me out on this: mRelative (relative-to-parent) is intialized with the bind-pose matrix for each bone, the FrameTransformationMatrix for each bone as loaded. Then mAbsolute is calculated for each bone is calculated to create a relative-to-root matrix. Then, for each bone, the offset matrix is calculated as the inverse of the bone's mAbsolute. Then, during animation, each bone's mRelative matrix (reused) is calculated by the animation controller from keyframe data. You then calculate each bone's mAbsolute (reused) iteratively as mRelative * parent.mAbsolute. The "final" array matrix (m_vMatrices) is set to bone-offset * bone.mAbsolute. m_vMatrices is used for animating the vertices.

 

If that's correct, your process looks okay - which you've pretty much confirmed by using tiny's offset matrices.


Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.


#7 Buckeye   Crossbones+   -  Reputation: 6407

Like
0Likes
Like

Posted 21 February 2014 - 11:15 AM

You're got me a little curious. I can't say I've ever compared calculations of the offset matrices. I do remember (from a long time ago) having trouble with tiny.x for some reason or other. However, somewhere I have Luna's skinned mesh code ("3D Game Programming .. DirectX 9.0c") examples which are very clear and use tiny.x. If I can locate the project, I'll add a routine to compare and let you know the results.

 

PM me. I may have another x-file you can try.


Edited by Buckeye, 21 February 2014 - 11:16 AM.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.


#8 Buckeye   Crossbones+   -  Reputation: 6407

Like
1Likes
Like

Posted 21 February 2014 - 03:11 PM

Okay, problem solved. And I should've thought of it before. I found some old project where I did the matrix calcs similar to what you're doing. When I calculated the bone offset, I then set the boneOffset = mesh to-root frame transform * calculated bone offset.

 

The mesh itself has a transform matrix by which the mesh vertices should be multiplied to put them in the mesh's parent space. Rather than multiplying each vertex by the mesh to-root frame transform, then multiply it by the bone offset, the offset matrix provided in the file (in the SkinWeights) is mesh-to-root-FrameTransform * boneCalculatedOffset.

 

You can think of it as meshVertex.toRoot * Inverse(boneToRoot). I.e., that matrix takes each vertex from mesh space to root space, then (through the inverse matrix) from root space into bone space. N.B., the mesh frame transform would your mAbsolute for the mesh frame.

 

So, if you want to do the operation yourself, get or calculate the to-root frame transform for the mesh. Then calculate the bone offset. Then set the "final" boneOffsetMatrix = meshToRootFrameTransform * boneOffset(calculated).

 

That results in the boneOffsetMatrix matching the offset matrix in the SkinWeights.


Edited by Buckeye, 21 February 2014 - 03:20 PM.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.


#9 Juliean   GDNet+   -  Reputation: 2755

Like
0Likes
Like

Posted 23 February 2014 - 04:15 AM

Ah, thank you very much for testing it out and explaining it to me! But if I understand this correctly, it means that I pretty much have no choice than to store the offset-matrices, since I want to support conversion from multiple file formats and not just x-files, and from what I get this is just how x-files handle things. Or am I mistaken and the "boneOffset = mesh to-root frame transform * calculated bone offset" is a general thing that every/most mesh formats would require for correct skinning?



#10 Buckeye   Crossbones+   -  Reputation: 6407

Like
1Likes
Like

Posted 23 February 2014 - 06:16 AM


the "boneOffset = mesh to-root frame transform * calculated bone offset" is a general thing that every/most mesh formats would require for correct skinning?

Oh, that 's a general statement! I wouldn't relate it to any particular file format, though. I'm not familiar with how other file formats export model data, but the principles of an animated skinned mesh have to hold true. Our discussion here was based on a particular x-file, but the rest was strictly math, with regard to rendering a skinned mesh.

 

Be aware, and once again, I'm not familiar with other formats particularly, it may be that the file is exported from the modeling program compensating for the mesh frame offset. That is, the vertex positions may be exported already relative to the root frame, and calculating the mesh-frame-to-root matrix is unnecessary. If so, that matrix is just the identity matrix. If you decide to use a different file format, you'll have to read the docs for that format, or study the data structure yourself, to find out how to extract equivalent data.


Edited by Buckeye, 23 February 2014 - 06:18 AM.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.


#11 Juliean   GDNet+   -  Reputation: 2755

Like
0Likes
Like

Posted 24 February 2014 - 03:59 AM

Allright, thanks again, I'll test some other formats and see how they work before I fixate on one solution ;)






Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS