Hello, I'm doing skeletal animation with smooth skinning. I've got a number of transformation matrixes and weighting factors per vertex that I use to transform them to object space from bind space. I'm a bit unsure what to do with normals. I read in some tutorials that they transform the normals using the same matrix as for the vertices but setting the normal's w to zero. This actually takes care of the translational part but what about scaling? I think bone transformation include scaling. How is scaling handled when transforming normals with smooth skinning? Thanks Markus

Setting the homogeneous co-ordinate w to zero means nothing more than to ignore the translational part of the transformation. Scaling is still active since it is located in the linear sub-matrix of the transformation, and that is apart from the translational part.

If you mean simply _direction_ vectors by the term "normal vector" (not the case, I assume) then all is ok. However, scaling a _orthogonal vector_ (just to use a more specific term) like the model is not in each case correct. It is correct if and only if the scaling factors are identical for each principal axis. If, on the other hand, you scale only in 1 direction (e.g. a height field) then scaling the normals the same way would yield in vectors no longer being orthogonal.

Example:
The vector
a := [ -1 , 1 ]
is orthogonal to the vector
b := [ 1 , 1 ]
since
a . b = -1 * 1 + 1 * 1 = 0
Scaling only in y direction by s:
a' . b' = -1 * 1 + s * s != 0
Scaling in both directions by s:
a' . b' = -s * s + s * s = 0

You could see from this 2D example that scaling the normal "orthogonal" to the model would do the trick:
a' . b' = -s * 1 + 1 * s = 0

Quote:Original post by haegarr
Setting the homogeneous co-ordinate w to zero means nothing more than to ignore the translational part of the transformation. Scaling is still active since it is located in the linear sub-matrix of the transformation, and that is apart from the translational part.

If you mean simply _direction_ vectors by the term "normal vector" (not the case, I assume) then all is ok. However, scaling a _orthogonal vector_ (just to use a more specific term) like the model is not in each case correct. It is correct if and only if the scaling factors are identical for each principal axis. If, on the other hand, you scale only in 1 direction (e.g. a height field) then scaling the normals the same way would yield in vectors no longer being orthogonal.

Example:
The vector
a := [ -1 , 1 ]
is orthogonal to the vector
b := [ 1 , 1 ]
since
a . b = -1 * 1 + 1 * 1 = 0
Scaling only in y direction by s:
a' . b' = -1 * 1 + s * s != 0
Scaling in both directions by s:
a' . b' = -s * s + s * s = 0

You could see from this 2D example that scaling the normal "orthogonal" to the model would do the trick:
a' . b' = -s * 1 + 1 * s = 0

May be my question wasn't written clearly.
With normal vector I mean the surface normal of a vertex. I do undestand the math. My question is more how to deal with that when doing skeletal animation with skinning.

Most tutorials transform vertices like that (example for 4 bones influencing the vertex:

v: vertex
v': transformed vertex
k0..k3: weighting factors for the influencing bones
M0..M3: transformation matrices for the influencing bones

v'= k0*(M0*v) + k1*(M1*v) + k2*(M2*v) + k3*(M3*v)

They do exactly the same transformation for normals with w=0 but renormalize n' after the calculation is done.

n: normal
n': transformed normal

n'=normalize ( k0*(M0*n) + k1*(M1*n) + k2*(M2*n) + k3*(M3*n) )

This should definately work when no scaling is taking place. Would the result with uniform scaling still be correct? Should'n the renormalizing be done for alle individual M*n terms instead of n'?

n'=k0*normalize((M0*n)) +k1*normalize((M1*n)) + k2*normalize((M2*n)) + k3*normalize((M3*n))

The previously posted example shows that non-uniform scaling does not keep orthogonality of normals. For this it is regardless whether the normal is thought for a vertex or else a face.

(If not state otherwise, in the following _uniform_ scaling is meant.)

Quote:Original post by muhkuh
n'=normalize ( k0*(M0*n) + k1*(M1*n) + k2*(M2*n) + k3*(M3*n) )

This should definately work when no scaling is taking place. Would the result with uniform scaling still be correct? Should'n the renormalizing be done for alle individual M*n terms instead of n'?

Three possibilities: (1) You have imported a model complete with normals from the disk, and you scale it by multiplying the vertex positions only (not the normals). (2) You have imported the model and scale also the normals. (3) You compute the normals on-the-fly from the scaled vertex positions.

For all three cases the cited formula is ok! For (1) and (3) the normals before and after the scaling does not differ at all. And for (2) here is the proove:
With s>0 denoting the scale factor,
normalize ( k0*(M0*n*s) + k1*(M1*n*s) + k2*(M2*n*s) + k3*(M3*n*s) )
= normalize ( s*k0*(M0*n) + s*k1*(M1*n) + s*k2*(M2*n) + s*k3*(M3*n) )
= s * normalize ( k0*(M0*n) + k1*(M1*n) + k2*(M2*n) + k3*(M3*n) ) / sqrt(s*s)
= n'
since the scale matrix (for _uniform_ scaling) is
S(s) := s * I
and so, for any concatenated transformation and s != 0
T * S(s) = T * s * I = s * T * I = s * T

The same proove demonstrates that there is no need for normalizing the individual parts due to uniform scaling. Moreover, when scaling plays no role, only rotation effects the normals, and rotations do not change the length of direction vector (aside from numerical precision issues).

Quote:Original post by muhkuh
n'=k0*normalize((M0*n)) +k1*normalize((M1*n)) + k2*normalize((M2*n)) + k3*normalize((M3*n))

This is generally wrong! The particular parts need not point into the same direction, and the weights are defined to sum to 1. So, this n' would in general not have unit length! Here a total normalization has to de done, and then, due to the stuff said above, you need the inner normalizations no longer.

Quote:Original post by haegarr
For all three cases the cited formula is ok! For (1) and (3) the normals before and after the scaling does not differ at all. And for (2) here is the proove:
With s>0 denoting the scale factor,
normalize ( k0*(M0*n*s) + k1*(M1*n*s) + k2*(M2*n*s) + k3*(M3*n*s) )
= normalize ( s*k0*(M0*n) + s*k1*(M1*n) + s*k2*(M2*n) + s*k3*(M3*n) )
= s * normalize ( k0*(M0*n) + k1*(M1*n) + k2*(M2*n) + k3*(M3*n) ) / sqrt(s*s)
= n'
since the scale matrix (for _uniform_ scaling) is
S(s) := s * I
and so, for any concatenated transformation and s != 0
T * S(s) = T * s * I = s * T * I = s * T

Thanks a lot for describing it so detailed. I understand that uniform scaling that is applied to the model as a whole doesn't affect normals. What I initially ment was uniform scaling coming from the individual bone and skin offset matrices so the s in the formula above would not be the same for all bone influences.

I read in several tutorials that the bone und skin offset matrices generally do not need scaling. Why not? It seems to me there is something I do not understand correctly about the bone and skin offset transformations. May be you or someone else can give me a hint:

1.) skin offset matrix:
I imagine this matrix to get the vertices from bind pose to local bone space like the opposite of taking a bone and moving it to the right place in the bind pose. This involves moving and rotating the bone (or the vertices) so that position of the joint and direction are correct. Until now I thought scaling would occur here too to get the bone from unit legth to it's desired length. This doesn't seem to be right. Why?

2.) bone matrix (the one transforming a bone relative to it's parent bone):
I thought this mostly consisted of rotations relative to a parent bone. But if there is not scaling in the skin offset matrix probably this is wrong too? Is the translation component in the bone matrix used to get the information about the length of a bone into the skeletal animation system?

So if there is is no scaling needed 99.9% of the time for the bone and skin offset matrices what are the 0.1%? Making individual body parts of the model grow or shrink?

Thanks a lot!

Markus

Perhaps I'm wrong, but I don't see why a skeleton animation systems should need scaling in common. The skeleton is imaginable as a hierarchy of bones, related together by kinematics. That is nothing other than related co-ordinate systems. The "length" of a bone is nothing other than the distance of the next local hinge from the origin of the current one (as you have asked in 2.).

Scaling the model needs to apply the scaling to the bones, too, or else the positions of the hinges would no longer be correct, of course. (Linear) translation is normally not done since it would mean to pull out limbs. Rotation is done, and it is done well due to forward kinematics. Since rotation does not alter distances, it does not pull out limbs. Scaling of individual parts is IMHO not needed, since it also would destroy the shape of the model (ok, some special effects like stretchable arms are out of scope here ;-).

Maybe the kind of skeleton system you have picked does some stuff I don't know, but I even don't see any need to scale bones from unit length to something other. You see its something vague what I told here. It seems that more details could be given only after some look at the tutorial...

Unfortunately I don't find the tutorial(s) at the moment. I read several and what I'm actually trying to is to get a base undestanding of what is going on in general and how to do a base implementation. So let's take this as a reference:
http://www.gamedev.net/reference/articles/article2221.asp

There are:

1.)local bone matrix (position a bone's joint relative to it's parent bone)
-translation (moving the joint away from the parent bone's joint, probably constant in different keyframes)
-rotation (likely to change in different keyframes)

2.)skin offset matrix (bring the vertices from bind to bone space)
-translation
-rotation

Is this correct?

Quote:Original post by muhkuh
http://www.gamedev.net/reference/articles/article2221.asp

Have had a look at it recently.

Quote:Original post by muhkuh
1.)local bone matrix (position a bone's joint relative to it's parent bone)
-translation (moving the joint away from the parent bone's joint, probably constant in different keyframes)
-rotation (likely to change in different keyframes)

This is what I meant above by the "forward kinematics" chain. A bone is defined w.r.t. its parent. As you stated, the translation is "probably constant in different keyframes". You are also right assuming the rotation to be the typical transformation in use for animation here. (Incorporating scaling here is possible, of course, but seldom done since it means to stretch bones.)

Quote:Original post by muhkuh
2.)skin offset matrix (bring the vertices from bind to bone space)
-translation
-rotation

This matrix does nothing else than defining a local co-ordinate system for the model, so that the model's mesh could be defined relative to the bone. E.g. the mesh could be made from a primitive instantiation routine, let's say it represents a sphere. Looking at the forearm bone, it origin could be placed into the shoulder, and the bone points in direction of the ellbow. Placing the sphere at the origin of the bone means to place it around the shoulder. Not good. Hence the "skin offset matrix" to be able to place (and orientate) the mesh where one wants. However, I would name this matrix "bringing the vertices from bone to mesh space".

So yes, IMHO you are (mostly) correct about this stuff. I would do it that way.

Scaling is used in skeletal animation systems. Well, not many out there support it though, as it can be quite anoying to implement support for it, and it can slow down runtime performance. Especially when dealing with non-uniform scale.

Think of it as animating the length of a bone in the chest of a character for creating a breath animation. Someone said he didn't see the difference between just translating the bone and scaling it, but visually it gives very different results on the skin. It won't change anything if you just render the skeletal hierarchy in lines of course. Then it would just look the same.

However, if you apply scaling to a bone, it has a different effect on the vertices linked to this bone, than if you would just move the bone to make it longer that way.

For example for a MMORPG scaling can be of great use when you like to customize the character on a per-bone level. So that you can make the legs longer, belly more fat, etc. The same goes for customizing the face of a character by using different bone scales.

Handling scale correctly however requires a more calculations, which makes everythign slower as well. And you have to take the scale into account in a lot of other places such as inverse kinematics routines. So definitely store your scale values separately so that you can easily work on a data set that has no scale, where you later on add the scale again.

About your question related to normals and smooth skinning. It depends on if you are dealing with non-uniform scale or not. If you deal with uniform scale, so where the x, y and z scale values are equal to eachother, then just multiply your normal with the 3x3 upper left part of your matrix. So only apply rotation (will also apply scale though as that is also part of the 3x3 part). Then let the hardware (or inside your vertex shader) normalize the normals before rendering. Don't normalize your normals on the CPU while skinning. That is a waste of CPU time, as the GPU can do it much faster.

When you deal with non-uniform scale, if I remember correctly you have to multiply your normals with the inverse transpose of the skinning matrix. So the transpose of the inverse.

As you can see this is one of those things that would slow down things, because you add scaling. I tell you, scaling is a biatch :) But it can be very very useful :)

Cheers,
- John

One last thing:

So usually the local Bone transformation matrix includes:
-rotation of the current bone relative to the parent bone
-translation away from the joint of the parent bone

This actually means the "length" of the parent bone is actually the translation component of the current bone's local matrix doesn't it? So the root bone usually has no translation in it's local matrix at all?

Thank you for this "talk to me like I'm a 3 year old" turorial :-)

Regards,
Markus