Sign in to follow this  
Darkbouncer4689

Animation implementation headache

Recommended Posts

Hey all,

I've spent the last few weeks trying to implement skin and bone animation in my engine with no success =(
I have a quick question for experienced animators out there.

I have my joints, inverse bind matrices, weights, and which joints effect each vertex all setup. I intended to ignore the animation data for now (input, output and interpolations) as I wanted to see my model transformed from model pose to the default pose before I added more complexity with animating it.

After hitting my head against the wall for days debugging, trying to figure out why I can render the skeleton properly but when I try to transform the model it explodes, I have to wonder if I am flawed in assuming that I can get a proper result while ignoring the animation data.

Is it necessary to pick a certain keyFrame from the animation data in order to render the model in a single default pose?

Thanks in advance.

Share this post


Link to post
Share on other sites
kauna    2922
[quote name='Darkbouncer4689' timestamp='1310876266' post='4836249']


Is it necessary to pick a certain keyFrame from the animation data in order to render the model in a single default pose?

Thanks in advance.
[/quote]


If you can provide a pose for the model otherwise then the answer is no. Otherwise, maybe it is a good start to use data which should work correctly.


There are lots of things that can go wrong with skin & bones. First of all, can you verify that your animation hierarchy is correct? I mean, if you render your bones are you getting the shape you are expecting?

Cheers!

Share this post


Link to post
Share on other sites
When I render my bones alone (no inverseBind calculations yet) I do get a correct skeleton.

When I try to render just my inverse bind matrices I get a crazy skeleton (i thought that the inverse bind matrices would be a skeleton of the character in bind pose, i.e. arms held outward with palms down)

Also once I multiply my bones with my inverse bind and try to render that skeleton it is also a crazy exploded mess.

I thought that maybe my problem was from multiplying in the wrong order, but I have tried every possible order and nothing looks right.

Any help is appreciated!

Share this post


Link to post
Share on other sites
kauna    2922
[quote name='Darkbouncer4689' timestamp='1310943882' post='4836529']
When I render my bones alone (no inverseBind calculations yet) I do get a correct skeleton.

When I try to render just my inverse bind matrices I get a crazy skeleton (i thought that the inverse bind matrices would be a skeleton of the character in bind pose, i.e. arms held outward with palms down)

Also once I multiply my bones with my inverse bind and try to render that skeleton it is also a crazy exploded mess.

I thought that maybe my problem was from multiplying in the wrong order, but I have tried every possible order and nothing looks right.

Any help is appreciated!
[/quote]

Hi,

Correct skeleton sounds good. You cannot render the inverse bone matrices and get something understandable out.

The inverse matrix is actually a matrix that transforms a vertex from the bind-pose to the bone space. This is needed in order to animate the skin correctly.

The point of bind-pose is to provide a common space for all the vertices. You can think that a vertex in the bind pose has already the bone matrix (bone matrices) applied once which you'll need to cancel with the inverse-bone matrix before animating it with a new bone matrix.

If the skeleton renders correctly, your animation data seems ok. The problem may be with the inverse transforms, skinning shader. Can you provide some code?

Share this post


Link to post
Share on other sites
Sure thing, here is my shader code

<script id="shaderFour-vs" type="x-shader/x-vertex">
attribute vec3 aVertexPosition;
attribute vec3 aVertexNormal;

uniform mat4 uMVMatrix;
uniform mat4 uPMatrix;
uniform mat3 uNMatrix;

attribute vec4 index;
attribute vec4 weight;
uniform mat4 Joints[44];

void main(void) {
vec4 newVertex;
vec4 newNormal;
int Index;
float totalWeight;
float normalizedWeight;


Index = int(index.x); //Cast to int
newVertex = (Joints[Index] * vec4(aVertexPosition, 1.0)) * weight.x;
totalWeight += weight.x;

Index = int(index.y); //Cast to int
newVertex = ((Joints[Index] * vec4(aVertexPosition, 1.0)) * weight.y) + newVertex;
totalWeight += weight.y;

Index = int(index.z); //Cast to int
newVertex = ((Joints[Index] * vec4(aVertexPosition, 1.0)) * weight.z) + newVertex;
totalWeight += weight.z;

Index = int(index.w); //Cast to int
newVertex = ((Joints[Index] * vec4(aVertexPosition, 1.0)) * weight.w) + newVertex;
totalWeight += weight.w;

if(totalWeight != 1.0)
{
normalizedWeight = 1.0 / totalWeight;
newVertex *= normalizedWeight;
}

gl_Position = uPMatrix * uMVMatrix * vec4(newVertex.xyz, 1.0);



For vertices with less than 4 joints effecting them, I use a weight of 0.

Share this post


Link to post
Share on other sites
kauna    2922
With a quick look to your shader code I noticed following:

[color=#1C2837][size=2]- [/size][/color][color=#1C2837][size=2]Index = [/size][/color][color=#1C2837][size=2]int(index.x);[/size][/color][color=#1C2837][size=2] <- are you sure that you get correct values from index? Are you able to pass the value to the shader as integer?[/size][/color]
[color=#1C2837][size=2]
[/size][/color]
[color=#1C2837][size=2]- you could normalize your weights while loading (just to simplify your shader)[/size][/color]
[size="2"][color="#1c2837"]
[/color][/size]
[size="2"][color="#1c2837"]- you could possible write the code as a loop which would make it a bit easier to understand and less prone to errors. [/color][/size]
[size="2"][color="#1c2837"]
[/color][/size]
[size="2"][color="#1c2837"]- it is hard to determine the order of the matrix calculations, perhaps you should have a another rigid mesh shader where you can verify that your view and projection matrix are applied in correct order.[/color][/size]
[size="2"][color="#1c2837"]
[/color][/size]
[size="2"][color="#1c2837"]The output of this shader is garbage more or less?[/color][/size]
[size="2"][color="#1c2837"]
[/color][/size]
[size="2"][color="#1c2837"]Cheers![/color][/size]

Share this post


Link to post
Share on other sites
Yeah I think the only way for me to debug it at this point is to write the code for it outside of the shader and debug each value. It's a lot of work but I've already spent about 30hours trying to figure out whats wrong. At least this will get be on the right path.

I suspect that my problem will be incorrect ordering of matrices.

Share this post


Link to post
Share on other sites
kauna    2922
Hi again,
I had a second thought. If you remove the skinning part of your shader (the lines where you use the Joints matrices)

[code]
newVertex = aVertexPosition;



//Index = int(index.x); //Cast to int
//newVertex = (Joints[Index] * vec4(aVertexPosition, 1.0)) * weight.x;
//totalWeight += weight.x;
...


[/code]


[size="2"][color="#1c2837"]You should end up with shader which shows your skin in the bind-pose if your projection and view matrices work correctly.[/color][/size]

[size="2"][color="#1c2837"]Cheers![/color][/size]

Share this post


Link to post
Share on other sites
I was able to render the model in bind pose (arms outward and palms down).

If I only render the joints I can also see the skeleton in a walking position.

It seems that those are about the only ways for me to debug things without pulling everything out of the shader =(

Share this post


Link to post
Share on other sites
kauna    2922
[quote name='Darkbouncer4689' timestamp='1310975209' post='4836694']
I was able to render the model in bind pose (arms outward and palms down).

If I only render the joints I can also see the skeleton in a walking position.

It seems that those are about the only ways for me to debug things without pulling everything out of the shader =(


[/quote]


Well I guess that we have narrowed down the problem to the skinning part of the shader (the code or the matrices have something wrong). Everything else seems to work as it should.

The index part is one question. You could try to multiply your [color="#880000"][font="CourierNew, monospace"][size="2"]index.xyzw by 255.0f before casting to int.[/size][/font][/color]
[font="CourierNew, monospace"][size="2"] [/size][/font]
[font="CourierNew, monospace"][size="2"][color="#880000"]Cheers!
[/color][/size][/font]

Share this post


Link to post
Share on other sites
JarkkoL    153
If you set all your joint matrices to identity, you should get the bind pose (i.e. the exported mesh should be in bind pose). That's a way you can check that your vertex shader is mostly working as expected.

It can be a bit tricky to get the joint matrices you send to the vertex shader right though and I'm guessing this is where the actual problem is. For the animation data I'm having joint->parent transformations, so after all the animation blending the final step before sending the joint transformations to vertex shader is to transform these animated joint->parent transformations to bind_space->joint->object_space transformations. The first step is to iterate through all the joints and multiply the animated joint->parent transforms with parent->object transforms to get joint->object transforms (the joints are sorted in parent->child order so parent is always transformed before being used for child transform to get the cumulative transforms). The next step is to iterate the transforms over again and multiply each calculated joint->object transform with its bind pose object->joint transform (i.e. joint inverse bind-pose transform) to get bind_space->joint->object transform, which you send to the vertex shader.

Also a small hint on optimizating the shader: You can first calculate weighted joint matrix and transform the vertex in the end with the matrix, instead of transforming the vertex with each joint and weighting the result. So your shader can be optimized to something like this:
mat=weight.x*Joints[int(index.x)]+weight.y*Joints[int(index.y)]+weight.z*Joints[int(index.z)]+weight.w*Joints[int(index.w)];
newVertex=mat*aVertexPosition;


Cheers, Jarkko

Share this post


Link to post
Share on other sites
[quote name='JarkkoL' timestamp='1310991832' post='4836771']
If you set all your joint matrices to identity, you should get the bind pose (i.e. the exported mesh should be in bind pose). That's a way you can check that your vertex shader is mostly working as expected.

Cheers, Jarkko
[/quote]

Interesting. So if I set all of my joints to identity and do the follow (assume bind shape matrix is identity also)
newVertex += vertex * inverseBindMatrix * weight

I should get a model in bind pose?

When I render the model geometry without changing anything the model has its arms outward, with palms facing down. This is called model pose right?

So then bind pose is what exactly? Would it be the model in the first keyframe of animation? Say in a walking position.

I tried what you said and I get a garbage model, so that may be where my problem lies if I am suppose to see a model that makes sense.

Thanks for the pointers!

Share this post


Link to post
Share on other sites
kauna    2922
[quote name='Darkbouncer4689' timestamp='1311042305' post='4837145']


Interesting. So if I set all of my joints to identity and do the follow (assume bind shape matrix is identity also)
newVertex += vertex * inverseBindMatrix * weight

I should get a model in bind pose?

When I render the model geometry without changing anything the model has its arms outward, with palms facing down. This is called model pose right?

So then bind pose is what exactly? Would it be the model in the first keyframe of animation? Say in a walking position.

I tried what you said and I get a garbage model, so that may be where my problem lies if I am suppose to see a model that makes sense.

Thanks for the pointers!
[/quote]

Bind-pose is the pose used to calculate the inverse bone matrices. Usually in the bind pose the mesh is as you describe (arms outwards) and the skeleton is in the same shape.
What you call model pose is actually the bind-pose.


Did you try to multiply the index by 255.0f ? Can you get a screen shot of the garbage?

Cheers!


ps.

[color=#1C2837][size=2]newVertex += vertex * inverseBindMatrix * weight <- the result of this code line would be garbage[/size][/color]

Share this post


Link to post
Share on other sites
JarkkoL    153
[quote name='Darkbouncer4689' timestamp='1311042305' post='4837145']
Interesting. So if I set all of my joints to identity and do the follow (assume bind shape matrix is identity also)
newVertex += vertex * inverseBindMatrix * weight
I should get a model in bind pose?
[/quote]
The line should be: newVertex += vertex * identityMatrix * weight;
Anyway, you shouldn't change the vertex shader like that but pass the identity joint matrices instead, to see that you pass & index them properly. Just an easy way to confirm that at least that part works as expected :)

[quote name='Darkbouncer4689' timestamp='1311042305' post='4837145']
When I render the model geometry without changing anything the model has its arms outward, with palms facing down. This is called model pose right?
[/quote]
That's called the bind pose.

Cheers, Jarkko

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this