Problems with Skeletal animation!

Started by
17 comments, last by Toji 18 years, 7 months ago
Ok, I have setup some code to test a simple example - based on the knowledge I have gained in this thread. It doesnt work just jet. The animated triangle changes shape and becomes longer and longer over the 10 frames. One of the 3 vertices are fixed at the same spot during animation. I was wondering if Im assuming wrong about what is what when I extract the info from a given file. Here is a dump from a hack I did to get a look inside the fileformat im using:

BB3D - 896 Bytes - version: 1
--TEXS - 43 Bytes - TexturePath: .\snowdrag.png
--BRUS - 47 Bytes
--NODE - 778 Bytes - Name: ROOT. Pos: (0.0, 0.0, -0.0). Rot: (1.0, 0.0, 0.0, -0.0).
----MESH - 180 Bytes - Master brush: -1
------VRTS - 132 Bytes - Flags: 0, 1, 2. 5 floats/vert.(x,y,z,u,v). Writen to Vertices.txt
------TRIS - 28 Bytes - Brush: 0 - Triangles: 2 - Writen to Triangles.txt and Pollies.txt
----ANIM - 12 Bytes - Flag: 0 - Frames: 30 - Fps: 0.0
----NODE - 517 Bytes - Name: joint1. Pos: (0.0, -20.0, -18.75). Rot: (-5.454082E-8, -2.909759E-8, 0.9567119, -0.29103667).
------BONE - 24 Bytes. Vertex1: 0 - Weight1: 1.0. Vertex2: 1 - Weight2: 1.0
------KEYS - 20 Bytes - Frame: 1 - Pos: (7.35381E-14, -20.0, -18.75).
------KEYS - 24 Bytes - Frame: 1 - Rot: (-5.454082E-8, -2.909759E-8, 0.9567119, -0.29103667).
------NODE - 370 Bytes - Name: joint2. Pos: (0.0, 0.0, -26.487024). Rot: (6.086863E-5, -1.576526E-4, 0.3604212, -0.9327897).
--------BONE - 0 Bytes.
--------KEYS - 68 Bytes - Frame: 1 - Pos: (2.939413E-10, -1.0051674E-6, -26.487024). - Frame: 10 - Pos: (5.8782906E-10, -2.0118482E-6, -26.487024).
--------KEYS - 84 Bytes - Frame: 1 - Rot: (6.086863E-5, -1.576526E-4, 0.3604212, -0.9327897). - Frame: 10 - Rot: (-4.9045168E-5, -1.617125E-4, 0.8582986, -0.5131507).
--------NODE - 139 Bytes - Name: joint3. Pos: (0.0, 0.0, -27.042791). Rot: (1.0, 0.0, 0.0, -0.0).
----------BONE - 24 Bytes. Vertex1: 3 - Weight1: 1.0. Vertex2: 4 - Weight2: 1.0
----------KEYS - 20 Bytes - Frame: 1 - Pos: (0.0, 0.0, -27.042791).
----------KEYS - 24 Bytes - Frame: 1 - Rot: (1.0, 0.0, 0.0, 0.0).


Im sorry if its a little messy, but I was posting it so I could be sure that I base the right matrices on the right info. This example is just like the one I posted in my initial post. 3 joints - like an arm!

Please take a look:

1. In each NODE I get a name, a position and a rotation. I have based my bindpose matrices on this info.

2. Each NODE contains 1 BONE chunk, 2 KEYS chunks and maybe child NODEs. I assume that the info in KEYS are for my lokal matrices. That I can turn into WorldMatrices multiplying them with their parents.
The KEYSchunks in joint1 and 3 contain only 1 keyframe, because there is no animation here. In joint2 there is more than 1 keyframe (like bending an albow).

What could be wrong? Anything strike you?

here is a specification of the fileformat im using:
http://www.zen54709.zen.co.uk/b3dformat.htm#BB3D
Advertisement
Well, it's really difficult to say, since there's a lot of things that can go wrong at this point, some of which may have little or nothing to do directly with your animation routine. My best bet, though, would be to check the order of operations on you matrix math. One screw-up there and you're whole system will fall apart. For example:

This is how a skinned mesh looks normally in my system.



All's well, right? Well, if I jump into the code and swap around the order that the matricies are multiplied in our recursive function, this is the result.



All it took for me to go from perfect skinning to a perfect mess was to flip the order of a single multiplication. Scary, huh?

So really, I guess the answer here is to play with it! Fiddle around with your order of operations and see what get's you closer to the result you want. Maybe you need to rotate a bone before you translate it, maybe you need to multiply this bones matrix by it's parent matrix instead of the other way around. It's all heavily dependant on the data you have and how your math libraries work, and the only way you'll figure it out is to experiment. It's not too easy, and its certainly one of the messier bits of programming, but it's part of the job description. ^_^

If, after all of that, you still have yet to get anything that looks even remotely right, come on back and we'll re-evaluate the situation.
// The user formerly known as Tojiro67445, formerly known as Toji [smile]
Oh my... I found some errors changed a little bit here and there. Im very close! My interpolation and rotation in general works perfekt, but something is wrong with the translation.

Take a look: www.animirage.dk

It sounds like the problem is related to a tip I read in Brett Porters "Skeletal Animation Tutorial".

It says: "Avoid using translations in joint transformations. You will probably end up stretching arms and legs instead of moving them, and it might end up looking funny. Think about it - if you move your elbow left, are you really moving the elbow or rotating the shoulder? If you are moving the elbow - its dislocated and you should either go to a hospital or a circus side-show. Rotate your shoulder instead."


When reading it sound obvious, but the program needs info about how far it is from its parent. So my conclusion is that I have to use translation somehow, but the way Im using it now is wrong!

I build my worldmatrices (animation and bindpose) based on local matrices with both translation and rotation. What on earth do I have to change??

Here is some of the code im testing with: (dont laugh - I manualy read in the quats and vecs from my writer-program)

[SOURCE]public static void createFinalMatrices(int frame) {  	  	//Manual setup of test-case.  	Matrix4f[] mAnim = new Matrix4f[3];  	Quat4f q1 = new Quat4f(-3.8316205E-8f, 0.9931783f, -0.11660568f, -4.85102E-8f);  	Quat4f q2 = generateQuat(frame); //here the interpolation between frame 1 and 10 occurs. (joint2)  	Quat4f q3 = new Quat4f(0.0f, 0.0f, 0.0f , 1.0f);  	Vector3f v1 = new Vector3f(0.0f, -5.5f, -17.25f);  	Vector3f v2 = new Vector3f(0.0f, 0.0f, -21.587032f);  	Vector3f v3 = new Vector3f(0.0f, 0.0f, -25.720858f);  	   	//joint1-root Worldmatrix.  	mAnim[0] = new Matrix4f(q1,v1,1.0f);  	  	//joint2 Worldmatrix.  	mAnim[1] = new Matrix4f();  	mAnim[1].mul(mAnim[0], new Matrix4f(q2,v2,1.0f));  	  	//joint3 Worldmatrix.  	mAnim[2] = new Matrix4f();  	mAnim[2].mul(mAnim[1], new Matrix4f(q3,v3,1.0f));  	  	//Multiplying all worldmatrices with the corresponding inverse World bindpose matrix.  	  	mAnim[0].mul(mInverse[0]);  	mAnim[1].mul(mInverse[1]);  	  	mAnim[2].mul(mInverse[2]);  	  	  	  	//Storing the array. Ready to use in the display-method.  	mFinal = mAnim;  }[/SOURCE]



Looking at the OGLFrames compared to MilkshapeFrames it looks like the middle joint is moving and that joint3 is moving further and further away from joint2. Im close - but no cigar!
Please help, if you have a clue about what to do next!

And btw, hope you get better! I read about your back-problems in your blog... :/
Ooh! Very close indeed!

Quote:My interpolation and rotation in general works perfekt, but something is wrong with the translation.


You're probably on the right track with this. I remeber that the last bug I worked out in my animation system was a translation bug. Everything looked as if it was rotating correctly, but it was translating in a slightly odd manner. What it turned out to be is that due to the order I was translating/rotating in each bone was translating by it's PARENTS translation, not it's own. This caused strange little discontinuities like my charecters waist not moving at all, even though the animation called for it to bounce up and down. (Since it was bound to the root node, which has no parent, it wasn't translating. Instead the chest was moving like the waist should have.)

So once again, it's probably an order of operations thing.

The bit that I would look at first is the order that operations are preformed in when you are passing in the rotation/translation to a new matrix. Does it rotate or translate first? That could make all the difference. I would do each step manually, which will give you much greater control and understanding of the situation than if you simply pass values to a constructor.

Just for examples sake, here's a snippit from my code that I use to build the world matrix:

void CSkeleton::BoneWorldTransform( int index, CMat4 *mat ){	if( index < 0 ) { return; } //We've reached the root node        int parent = animBones[index].parent;	(*mat) = (*mat) * animBones[index].rot.ToMat4();	mat->Translate( animBones[index].pos );			BoneWorldTransform( parent, mat );}


Obiviously this is just a simple recursive function that works its way up the skeletal tree to generate a final matrix for whatever bone index you passed in initially. If I flip the order of that rotate and translate then the animation comes out looking "almost right", much like yours. I should point out, though, that this particular function starts at a branch of the tree and works it's way to the root, where as you're going the opposite direction. That's not a problem, either one works, but you'll probably have to do the math "backwards" from how I do it.

In any case, try doing the rotation/translation seperately and see if that gives you better results. Other than that, I think you've got it down!

Quote:And btw, hope you get better! I read about your back-problems in your blog... :/


Wow, somebody actually reads that thing? Go figure... ^_^ Thanks for your well-wishes! It's more of an inconvience than anything as long as I stay medicated, but it's still a pain (pun intended!)



// The user formerly known as Tojiro67445, formerly known as Toji [smile]
Lost in translation....=)
Naah, its not that bad though I havent seen the light yet. I read your last post and I have gained lots of insight testing and trying every setup I could think of.
Quote:Does it rotate or translate first? That could make all the difference. I would do each step manually, which will give you much greater control and understanding of the situation than if you simply pass values to a constructor.

I have been looking into it and I found that the constructor builds the matrix translation first.. then rotation. I came to this conclusion making 2 matrices, one with rotation first and one with translation first... last one was equal to the one the constructor created.
To be quite honest I was happy about this observation. You seemed to have seen the light rotating first. So I thought: theres the bug!=)

The next thing I did was seting up test after test (joint3 worldmatrix). here they are:

Test1: joint1(trans)*joint1(rot) * joint2(trans)*joint2(rot) * joint3(trans)*joint3(rot)
This is the test you have allready seen - almost right it seems!

Test2: joint1(rot)*joint1(trans) * joint2(rot)*joint2(trans) * joint3(rot)*joint3(trans)
This setup created a counterclockwise movement going towards the left like test1, but with a larger curve. I had to zoom out a bit to see it.

Test3: joint3(rot)*joint3(trans) * joint2(rot)*joint2(trans) * joint1(rot)*joint1(trans)
As I see it this test is a setup exactly like yours. Result: Damn! Large clockwise rotation towards the right. Not even close!

Test3: Just for the fun of it I transformed the vertices of the moving triangle with the finalmatrix of joint2 (not joint3). Result: VERY CLOSE - the point of the moving triangle has the same position during the animation - in other words: the tip/point of the triangle is the center of rotation - this is what we want!!! BUT..... its rotating clockwise - NOT counterclockwise as we want. Have you got a clue about whats going on?

Test4: joint3(trans)*joint3(rot) * joint2(trans)*joint2(rot) * joint1(trans)*joint1(rot)
Large rotation clockwise towards the right. Not even close.

---

Though all 4 test-cases gave me a wrong result, I have learned alot and I have a lot more insight than I had when I wrote my last post in this thread.

Observations
1. In all testcases except test3 the final matrices of joint2 and 3 are the same. To begin with I thought this was an error but after a long walk around the lake reflecting, I came to a the conclusion that its maybe not that wierd after all. Since Im inverting with the inverted bindpose all thats left is the rotation from joint2 that follows along to joint3.... Im not at all sure, so your words on this would be greatly apreciated...

Quote:(*mat) = (*mat) * animBones[index].rot.ToMat4();
mat->Translate( animBones[index].pos );

2. In your method BoneWorldTransform I assume that you pass in an identity-matrix in your initial call? Also, I see that you multiply the rotationmatrix to the WorldMatrix you are building. What you do with the translation-part is unclear... This could be where im wrong. What exactly do you do here: mat->Translate( animBones[index].pos ); I take an identity matrix setTranslation to it and multiply it just like I do with the rotation-part.

3. The Milkshape program uses a coordinate-system that is the mirror coordinate-system of the one used in Blitz Basic 3d. I can see that the Milkshape system is also mirror to the openGL system, hence blitz3d and OpenGL should have the same orientation of the coordinate-system. So when exporting to blitz3d format, I should get a format that works with OpenGL. Do you agree with this conclusion? Im just trying to find the bug...=)

4. Reflections: I have only 3 translation vectors and 3 rotation Quats... There are not too many combinations! guess thats the most annoying part..

Quat4f q1 = new Quat4f(-3.8316205E-8f, 0.9931783f, -0.11660568f, -4.85102E-8f);
Quat4f q2 = generateQuat(frame);
Quat4f q3 = new Quat4f(0.0f, 0.0f, 0.0f , 1.0f);
Vector3f v1 = new Vector3f(0.0f, -5.5f, -17.25f);
Vector3f v2 = new Vector3f(0.0f, 0.0f, -21.587032f);
Vector3f v3 = new Vector3f(0.0f, 0.0f, -25.720858f);

So here is the logic I have come up with:
The first thing that happens is a translation from the worldsystems (0,0,0) to the location of joint1. v1 gives us that translation. Now we are at the position of joint1 and next we rotate (q1) to orient the z-azis in the direction of joint2. That way we only have to translate the z-azis (v2) to get there. At joint2 animations occurs (again here the z-azis are oriented so we only have to translate the in the z-azis direction) and dependent on what frame we are in we translate to som position. The logic I have come up with here kinda goes against the "rotate first"-theory, so I must be missing something... Please comment on this!

----

Still no cigar, but im not giving up... =/ I think the man upstairs is testing my patience... ;)

(Check out the number of views of this thread..! Seems to be quite a lot of interest on this subject)
Heh... Test3 had a large clockwise rotation and modifying it a bit gave the right small-scale rotation in the wrong direction (clockwise). Wondering about that I thought maybe I can do that with test2 that has a large rotation in the RIGHT direction -counterclockwise. I toke out all translation in the creation of the worldmatrices and there it was. It looks right - only problem is that the center of rotation is not exactly at the tip of the triangle... 2mm's off or so... Im thinking that maybe this is it!
I was thinking about Brett Porters tutorial when I got the idea. Maybe some formats dont require translation in the creation of the worldmatrices? I dont know... I will check this setup and make sure it holds water ( Im not sure ). Those 2mm annoys me..

What ever the case is I will get back to you either extremely happy or extremely back on the school-bench!!! =)

--------------

EDIT: Didnt work obviously... It was pure luck that the desired center of rotation was close to the origin of the bindpose-coordinatesystem!
Please read the post above this one!

[Edited by - ASL on September 6, 2005 12:58:16 PM]
Hm... well this is tricky. As far as I can tell you've tried every logical combination there is. You may want to check and make sure that all of the information it's giving you is local. I've heard of formats before where the rotation is local but the translation is given in world coordinates. Why anyone would do that is beyond me, but I guess it could happen.

Logically stepping through this as if the coordinates you gave were running through my code, here's what you would get:

(Psudocode-ish)
FinalMat[0] = IdentityMat();FinalMat[0] *= Joint[0].Rot;FinalMat[0] *= Joint[0].Trans;FinalMat[1] = IdentityMat();FinalMat[1] *= Joint[1].Rot;FinalMat[1] *= Joint[1].Trans;FinalMat[1] *= Joint[0].Rot;FinalMat[1] *= Joint[0].Trans;FinalMat[2] = IdentityMat();FinalMat[2] *= Joint[2].Rot;FinalMat[2] *= Joint[2].Trans;FinalMat[2] *= Joint[1].Rot;FinalMat[2] *= Joint[1].Trans;FinalMat[2] *= Joint[0].Rot;FinalMat[2] *= Joint[0].Trans;FinalMat[0] *= InverseMat[0];FinalMat[1] *= InverseMat[1];FinalMat[2] *= InverseMat[2];


Does that seem to match up with what you're getting?

One other thing you may want to try: For me it was a lot easier to render a simple line-based skeleton than the full blown skinned charecter, and I had a period of time where I had a skeleton that I could see was animating perfectly, but the skin wasn't matching up. I would recommend trying to render the skeleton first, then dive into getting the polygons to follow.

To render the skeleton you're simply going to take the local position for each joint and multiply it by the world matrix of that joints parent. I should note that when calculating the skeleton positions you DON'T want to multiply by the inverse bind pose. These points are already in local space, so it's not nessicary. In any case, that will give you an array of points that represent to positions of all the joints in your skeleton. By drawing a line from a joint's position to it's parents position, you'll get a simple but effective "skeletal view" of your animation. Try that and see what you get. If things look off at that low level nothing-but-bones stage then you'll need to correct them before you can ever get to the actually skinning.


// The user formerly known as Tojiro67445, formerly known as Toji [smile]
WOOOOOHOOOO!!!
Another 3 weeks, and I found the bug.
But man, it was a dumb bug.

In my filewriter program I had a "smart" scale funktion. So I could scale size of polygons up and down. Only problem was that I had not applied the funktion to the translation info.

All these weeks I had never thought of checking the info that came from my writer program. I found the bug like this:

I had ordered the book "Advanced Animation with DirectX" from Amazon and it came yesterday. This book is centered around the .x file format that also contains translation and rotation in the form of a transformation matrix - 16 float array. I got the idea to export the milkshape file as a directx file. That way I could see if I did the calculations to create the local matrices right at least. Everything was the same except, the translation part of the matrix was 0.1 less than mine. It didnt take long to find out where the bug was after this. See I have had my scale on 0.1, all this time.

Anyways, everything you said was right and it works. Im soo pleased, you are the man!!
To everyone who reads this give him some rating, he deserves it! After all, skeletal animation is the most advanced animation technique on the market today. Much more complicated than keyframed animation.
If anyone plan to follow my footsteps and learn skeletal animation I have some books I can recommend (they are a scarce commodity):

"Advanced Animation with DirectX" by Jim Adams (our very own moderater here on Gamedev)
"Focus on 3d models" by Evan Pipho (contains a good milkshape example and more)

You dont find much more on the subject. In any case, I want to give back what I have been given by Toji and others. So if anyone needs my help on the subject feel free to pm me.



Awesome! I knew you'd get it working eventually! Give yourself a big pat on the back: YOU deserve it!

Quote:But man, it was a dumb bug.


They always are. ALWAYS. It's just part of how a programmers brain works, I think. We have the smarts and foresight to design our data structures from the ground up with optimization in mind, but we completely miss the fact that we happened to forget to initialized varible xyz to zero. >_< I call this the programmers blind spot. We tend to assume everything below a certain complexity level is correct, and therefor don't check them at all.

Quote:See I have had my scale on 0.1, all this time.


Yup, that would do it. ^_^

Quote:Anyways, everything you said was right and it works. Im soo pleased, you are the man!!


While I appreciate the gratitude, you give me WAY too much credit. After all, I'm not the one who just spent three weeks busting my brain to get it working!

Quote:In any case, I want to give back what I have been given by Toji and others.


Glad to hear it! That's the true programmers spirit!

If you have any further questions then don't hesitate to ask, but it seems to me like you've got it all sorted out! Glad to hear it, man! Have fun with your newly animated programming advetures!
// The user formerly known as Tojiro67445, formerly known as Toji [smile]

This topic is closed to new replies.

Advertisement