Jump to content

  • Log In with Google      Sign In   
  • Create Account


Quaternions and Animation Blending questions


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
12 replies to this topic

#1 Tispe   Members   -  Reputation: 982

Like
1Likes
Like

Posted 09 July 2013 - 03:57 AM

Hi

 

I have come to that point in time where I need to implement Animation blending. And I need help to clearify the process.

 

Right now I am skinning on the GPU. When my character is drawn the vertices gets transformed by weighted bone matrices in the vertex shader. At the moment my character is in the default T-Pose. Before drawing, each bone is multiplied by the parent(top-to-bottom) on the CPU and the entire combined skeleton is passed to the GPU. So all of that is in place. All that remains is to animate these T-Pose matrices, which I need help to do.

 

It is clear to me that I need to animate the bones before combining them to the final skeleton which is passed to the GPU.

 

I have an array of bones (D3DXMATRIX) in T-Pose which is not combined. I can animate ontop of the T-Pose by multiplying each bone (in T-Pose) with an animation key(matrix retrieved from a Quaternion). If I then combine the bones to the final skeleton and pass it to the GPU I get a pose at that exact animation key.

 

To blend between two animation keys I use the D3DXQuaternionSlerp function on two animation keys(only the Quaternion part of the key). The animation track(keys) are stored in structs as 'Quaternions, Vec3 Position and Vec3 Scale'. From the result I extract the D3DXMATRIX and apply it to the appropiate T-Pose bone. When all bones are animated I can combine them and pass the skeleton to the GPU and render.

 

All this is good for One Animation track. But I want to Blend several animations. Now how do I do this? I which order do I blend the different tracks? Does it matter?

 

Say I have 3 Animation Tracks I want to blend. I will use the D3DXQuaternionSlerp three times, for two keys in each aniamtion track. Now I have 3 Quaternions (one from each track), what should I do with these to get the D3DXMATRIX to apply to my T-Pose bone? And how do I weight them?

 

Cheers!


Edited by Tispe, 09 July 2013 - 04:02 AM.


Sponsor:

#2 Norman Barrows   Crossbones+   -  Reputation: 1966

Like
0Likes
Like

Posted 10 July 2013 - 10:16 AM

ok, first off, there appear to be two type of animation "blending" used in games:

 

1. animation "combining". here, different limbs have their own animations, such as "left arm raise shield", "right arm swing sword", and "body move forward". these are combined to move the appropriate bones to result in a single "move forward, raise shield. and swing sword" animation, without having to make separate animations for each combo of possible limb movements..

 

2. animation "averaging". here, two complete animations (such as "walk" and, well i can't think of a practical example, so lets use "stand") are averaged together (by averaging their key frames most likely), resulting in a complete new animation that's the average of the original two. averaging a "walk" and a "stand" animation would result in a less pronounced walk animation. 

 

i have seen animation combining mentioned in reference to high end titles. i haven't seen much or anything of true animation averaging mentioned anywhere. i'm not sure its all that useful. or maybe i just can't think of an application offhand.

 

it sounds like you're interested in animation combining, correct?


Norm Barrows

Rockland Software Productions

"Building PC games since 1988"

 

rocklandsoftware.net

 


#3 Tispe   Members   -  Reputation: 982

Like
0Likes
Like

Posted 10 July 2013 - 02:14 PM

I am interested in combining yes. For example, the "Walking" animation track only control the legs and lower torso. The "Swing Weapon" animation track controls upper torso, arms and head.

 

I believe the "walking" track only contains the key frames for the bones it controls.

 

One extra caveat I is that the animation tracks seems to be only be relative to the T-Pose. Such that the animation tracks perturb the T-Pose. Rather than scrapping the T-Pose and directly determine every bone from the track. I'm not sure if this is the normal way to do it, but it is what I have.

 

I have looked around and the only thing I can find on this subject is some papers on "Dual Quaternion Skinning" where they seem to just multiply the dual-quaternions together. But I am new and alot of it seems beyond my understanding.



#4 Sector0   Members   -  Reputation: 456

Like
1Likes
Like

Posted 10 July 2013 - 02:27 PM

Hi Tispe,

Obviously the quaternion multiplication is not commutative this means q1 * q2 is not equal to q2 * q1. beside this you might know that quaternion slerp always interpolates between two rotations on the shortest arc from all possible arcs between them so using slerp is a great idea to achieve animation blending.

 

For blending, first you have to order your animations in the way you need. For example you can enable walk animation firstly and next run animation could be enabled. I recommend you to always save the order of  enabled animations in a queue. Users have to enable them in order they want.

Each bone for each animation owns a weight. In animation systems there are two ways to apply weights for blending. First one is mixing and the second is additive weighting.

 

Mixing: For mixing you should calculate the average weight of each animation with respect to the other animation weights. For example if you want to blend three animations together, they should be blended like this (assume that anim1, anim2 and anim3 have been enabled in order):

 

if (anim1.weight > anim2 weight)

{

q = slerp( anim1, anim2, 1- anim1.weight/(anim1.weight + anim2.weight));

}

 

else

{

q = slerp( anim1, anim2, anim2.weight/(anim1.weight + anim2.weight));

}

 

if ( anim1.weight + anim2.weight > anim3.weight >)

{

q = slerp( q, anim3, 1- (anim1.weight + anim2.weight)/(anim1.weight + anim2.weight + anim3.weight));

}

 

else

{

q = slerp( q, anim3, (anim3.weight)/(anim1.weight + anim2.weight + anim3.weight));

}

 
After this, you can multiply q to the T-pose of the current bone: qfinal = q * T-pose
It is better to make a loop for this procedure because there maybe many animations blending with each other.
 

Some graphics engines like OGRE do not blend different animations with slerp interpolation like i mentioned above. They sequentially multiply quaternions in order. For

example three animations are calculated like this (assume that anim1, anim2 and anim3 have been enabled in order):

 

anim1.weight = anim1.weight/(anim1.weight + anim2.weight + anim3.weight)

anim2.weight = anim2.weight/(anim1.weight + anim2.weight + anim3.weight)
anim3.weight = anim3.weight/(anim1.weight + anim2.weight + anim3.weight)

 
q = (anim3.weight * q3) * (anim2.weight* q2) * (anim1.weight * q3) ;
 
This method does not provide accurate blending resultd because it does not interpolates on the shortest arc between 2 rotations but its result is acceptable and it has a better performance in comparison to using slerp for blending.
 
You might want to implement two techniques and use them for different usages. For example if you want to have animations with lower LOD you can apply the second technique to them.
 
 
Additive weight computation:  Additive animations are one type of animations that can be added with actual weights to other animations. Their weights should not to be averaged. They are used for asynchronous usages like a simple hit reaction. You can assume a simple noise animation presenting bullet hits. This noise animation can be added to any animation asynchronously. Your character can be shot in walk, run , idle or may other possible animations. This noise can be added up on the animations to show that your character has been shot while running or walking and so on. So for additive animation blending we do not need to average the weights with respect to other animations. The transforms of an additive animation is applied with its actual weights. Just note that the additive animations have to be added at the final step this means you have to calculate the mixing animations and then apply the additive animation like this:
 
Final blended rotation = additiven * additiven-1 * additiven-2 * ... * additive* MixedAnimations
 
For each animation you have to save its blending type to know whether it is average or additive to apply its weights with the proper technique.
 
I have two short posts on my blog about unit quaternions. you might want to check them out:
 
 
Hope this would help.

Edited by Sector0, 10 July 2013 - 03:33 PM.


#5 Tispe   Members   -  Reputation: 982

Like
0Likes
Like

Posted 10 July 2013 - 03:42 PM

Final blended rotation = additiven * additiven-1 * additiven-2 * ... * additive1 * MixedAnimations

Is this quaternion multiplication or slerp?

D3DXQUATERNION q_t_pose = GetTPoseQuaternion(BoneID);
D3DXQUATERNION q1 = GetQuaternionFromAnimation(Walking, BoneID, time);
D3DXQUATERNION q2 = GetQuaternionFromAnimation(Shooting, BoneID, time);
D3DXQUATERNION q3 = GetQuaternionFromAnimation(Hit, BoneID, time);

D3DXQUATERNION qFinal = q3 * q2 * q1 * q_t_pose;

D3DXMATRIX BoneMatrix;
D3DXMatrixRotationQuaternion(&BoneMatrix, &qFinal);

//Combine all BoneMatrices to a final skeleton for use for skinning vertices

Would this work?

 

Is it possible to only use additive animations on top of T-Pose?


Edited by Tispe, 10 July 2013 - 03:42 PM.


#6 Tispe   Members   -  Reputation: 982

Like
1Likes
Like

Posted 11 July 2013 - 05:23 AM

A paper describing Dual Quaternion Skinning http://webir.tcd.ie/bitstream/2262/64125/1/sdq.pdf has some interesting solutions. Though I find it very hard to grasp.

 

What it does is that instead of passing 4x4 Bone matrices to the GPU, it passes 2x4 matrices containing Dual Quaternions. On the Vertex Shader the appropiate Dual Quaternions are weighted and added together to form the final Dual Quaternion. Which in turn is converted to a matrix that transform the vertex.

 

So here are some new questions:

Is it possible to apply the same weighting and addition used for skinning, to add together Animation Tracks?

 

Currently a T-Pose bone is built by converting a quaternion to a matrix, and building a translation matrix from a position vector, these are then multiplied to form the T-Pose bone. I could convert the T-Pose from matrix form to Dual Quaternion form, but can I build a Dual Quaternion directly from a quaternion and a PosVector?

 

So, is this how I should Animate characters?

CPU:

1. Build T-Pose bones in Dual Quaternion form.

2. Create a list of animation tracks to blend and set a weight for each track.

3. For each bone, enter every animation track and get the Bone Quaternion at time t by Slerp(key1, key2, t'),  convert it to Dual Quaternion, weight it and add it to T-Pose bone.

4. Combine all T-Pose bones which are now animated and send the skeleton to the GPU.

VS:

5. Use Dual Quaternion skinning to weight and add influencing bones and build a transformation matrix.

6. transform vertex by matrix, then by world matrix, then by view matrix and finally by projection matrix.

 

Am I thinking right?

 

 

EDIT:

I found an article describing "nlerp" http://number-none.com/product/Understanding%20Slerp,%20Then%20Not%20Using%20It/ . Supposedly it is more CPU friendly than Slerp and supposedly it is commutative, meaning I can blend the animation tracks in any order with the same results. Anyone here have experience with nlerp?


Edited by Tispe, 11 July 2013 - 08:48 AM.


#7 Sector0   Members   -  Reputation: 456

Like
0Likes
Like

Posted 12 July 2013 - 03:09 AM

 

Is this quaternion multiplication or slerp?
D3DXQUATERNION q_t_pose = GetTPoseQuaternion(BoneID);
D3DXQUATERNION q1 = GetQuaternionFromAnimation(Walking, BoneID, time);
D3DXQUATERNION q2 = GetQuaternionFromAnimation(Shooting, BoneID, time);
D3DXQUATERNION q3 = GetQuaternionFromAnimation(Hit, BoneID, time);

D3DXQUATERNION qFinal = q3 * q2 * q1 * q_t_pose;

D3DXMATRIX BoneMatrix;
D3DXMatrixRotationQuaternion(&BoneMatrix, &qFinal);

//Combine all BoneMatrices to a final skeleton for use for skinning vertices

 

 

This is quaternion mutiplication. Actaully this means that first you rotate your object with q_t_pose then with q1 next q2 and q3 respectively. This can be used for additive animation bledning but is not suitable for mixing animations.

 

 

 

So, is this how I should Animate characters?
CPU:
1. Build T-Pose bones in Dual Quaternion form.
2. Create a list of animation tracks to blend and set a weight for each track.
3. For each bone, enter every animation track and get the Bone Quaternion at time t by Slerp(key1, key2, t'),  convert it to Dual Quaternion, weight it and add it to T-Pose bone.
4. Combine all T-Pose bones which are now animated and send the skeleton to the GPU.

 

 

This procedure is OK as far as you normalize the weights. For the dual quaternions i have to say that i've never used it and know a little about it. I always use quaternions and it responds very well for rotations on CPU. I think you can use quaternions in CPU computations for animation blending. I have no idea about GPU computations.

 

 

I found an article describing "nlerp" http://number-none.com/product/Understanding%20Slerp,%20Then%20Not%20Using%20It/ . Supposedly it is more CPU friendly than Slerp and supposedly it is commutative, meaning I can blend the animation tracks in any order with the same results. Anyone here have experience with nlerp?

 

 

I have worked with nlerp before. I prefer to use nlerp when i want the animation LOD to be low. nlerp takes less computation. It can be calculated like this:

 

nlerp(q1, q2, t) = (1-t ) * q1 + t * q2;

 

Slerp is more accurate but it takes more computations because it has to extract the quaternions and calculate many sin, cos, arccos and again multiply them. You can consider this example for using slerp:

 

Lets assume that you want to create a side scroller action shooter and you have 3 animations for aiming. One is aiming 90 degrees up, other is aiming forward with 0 degree with respect to horizon and the third is aiming 90 degrees down. You want to blend between these three animations for character aiming. You get the mouse world position and calculate its difference with your character's world position. Then you calculate the deviation by using the vector dot multiplication of the resulted aim direction and character forward vector. The result shows that the deviation degree is 45. Now you want to blend between aim_up_90 and aim_forward animations so your character can target the mouse cursor with 45 degrees with respect to horizon. If you use slerp blending like this:

 

Slerp (aim_forward_key, aim_up_90_key, 45/90)

 

You will catch accurate results. Your character aims at 45. But if you use nlerp your character never looks at 45 degree. For example he might look at 30 degrees while the mouse aim is at 45 degrees. So it would be great to use nlerp for the purposes which the accurate blending is not really matters.

 

In other words, by using quaternion Slerp you can rotate one vector to another linearly with respect to degrees between them.


Edited by Sector0, 12 July 2013 - 07:35 AM.


#8 RobTheBloke   Crossbones+   -  Reputation: 2324

Like
0Likes
Like

Posted 12 July 2013 - 05:21 AM

 

This procedure is OK as far as you normalize the weights. For the dual quaternions i have to say that i've never used it and know a little about it. I always use quaternions and it responds very well for rotations on CPU. I think you can use them at least in CPU computations. I have no idea about GPU computations.

 

 

 

Dual quaternion skinning has nothing to do with animation blending. It's just an alternative geometry deformation technique.

 

 

I found an article describing "nlerp" http://number-none.com/product/Understanding%20Slerp,%20Then%20Not%20Using%20It/ . Supposedly it is more CPU friendly than Slerp and supposedly it is commutative, meaning I can blend the animation tracks in any order with the same results. Anyone here have experience with nlerp?

 

 

I have worked with nlerp before. I prefer to use nlerp when i want the animation LOD to be low. nlerp takes less computation. It can be calculated like this:

 

nlerp(q1, q2, t) = normalise( (1-t) * q1 + t * q2 );

 

FTFY


Edited by RobTheBloke, 12 July 2013 - 05:24 AM.


#9 Tispe   Members   -  Reputation: 982

Like
0Likes
Like

Posted 12 July 2013 - 06:43 AM

I notice that Dual Quaternion skinning follows a non-commutative weighted sum, not lerp nor slerp.

 

However on animation blending, is it possible to nlerp Dual Quaternions commutativly?

 

There is alot of things to keep track of here. Because there are many quaternion operations going around, and doing them correct seems important.

 

1. Inside the Animation Track, Slerp between two quaternion keys seems OK. Outputs a Quaternion, possibly convert to Dual Quaterion.

2. Outside when blending animations, One Quaternion/Dual Quaterion from each track must be blended, either nlerp, or Slerp, or weighted multiplied or weighted summed. Which one is right?

3. How are the blended animations applied to the T-Pose bone, nlerp, Slerp, multiplication, summation?

4. When combining Dual Quaternion Bones to a Full skeleton,  nlerp, Slerp, multiplication, summation?

5. On the GPU, Dual Quaternion Skinning, in which I see papers use non-commutative weighted sum.

 

There are many, many possiblities.

 

To me it looks like:

1. Slerp keys inside a track. Output a quaternion.

2. nlerp all quaternions. q = nlerp(q1, nlerp(q2, nlerp(q3, q4)))     //any order is ok because nlerp is commutative. notice q does not have translation properties

3. qfinal = q* qT-pose.       //T-Pose Bone is a Dual quaternion, but q is a quaternion, perhaps possible to overload operator 

4. for each combine(qfinal, qparent);

5. qskinning = q1w1 + ..... + qnwn;

 

If the animation track also has translation then that must be implemented somehow, perhaps with Dual Quaternions from the track. But then which procedure to use?


Edited by Tispe, 12 July 2013 - 07:45 AM.


#10 Sector0   Members   -  Reputation: 456

Like
0Likes
Like

Posted 12 July 2013 - 07:53 AM


nlerp(q1, q2, t) = normalise( (1-t) * q1 + t * q2 );

 

Thanks for fixing. I forgot to say that the result of nlerp between two unit quaternions is not a unit quaternion and the result has to be normalized.



#11 Paradigm Shifter   Crossbones+   -  Reputation: 5214

Like
1Likes
Like

Posted 12 July 2013 - 08:30 AM

A lerp between 2 quaternions isn't normalised. Take a wild guess at what the 'n' in nlerp stands for ;)


"Most people think, great God will come from the sky, take away everything, and make everybody feel high" - Bob Marley

#12 Tispe   Members   -  Reputation: 982

Like
0Likes
Like

Posted 12 July 2013 - 03:09 PM

Hi again

 

It turns out that my animation track keys has a Quaternion for rotation, and two vectors for Translation and Scaling. Can I blend each part by themselves? i.e

q = nlerp(q1, q2);
t = t1*t2; 
s = s1*s2;

D3DXMATRIX BlendedBone;
D3DXVECTOR3 center(0.f,0.f,0.f);
D3DXVec3TransformCoord(&center, &center, &TPoseBone);
D3DXMatrixTransformation(&AnimatedBone, &center, &q, &s, &center, &q, &t);

D3DXMATRIX AnimatedBone = TPoseBone * BlendedBone;

Or is there a way to do all of this without matrices? I am thinking dual quaternions, but then I have to ignore scaling.



#13 Sector0   Members   -  Reputation: 456

Like
0Likes
Like

Posted 13 July 2013 - 07:29 AM


q = nlerp(q1, q2);
t = t1*t2;
s = s1*s2;

 

Use Lerp for blending instead of vector cross:

 

t = Lerp(t1, t2, a);

s = Lerp(s1, s2, a);


Edited by Sector0, 13 July 2013 - 07:31 AM.





Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS