GPU Skinned Skeletal Animation Tutorial

Started by
3 comments, last by spacerat 10 years, 1 month ago

Today I completed a skinned skeletal animation tutorial, which is very helpful if you are just about to start with game development.

Different from the other tutorials I found in the web, this one is very light weight ( < 800 lines for the main mesh & animation code ) and works well with most modeling environments.

Summary

It has the following properties / features:

  • GPU Skinning / Matrix Palette Skinning
  • Bump Mapping (automatic normal map generation)
  • Spheric environment mapping
  • Ogre XML Based
  • Shaders in GLSL
  • Visual Studio 2010
  • Select LOD level with F1..F5

It is ready to use, which means you can load and display the animated models in a few lines of code:


static Mesh halo("halo.material",//	 required material file)
		 "halo.mesh.xml",//	 required mesh file
		 "halo.skeleton.xml");// optional skeleton file

int idle = halo.animation.GetAnimationIndexOf("idle");

halo.animation.SetPose(idle,	   // animation id (2 animations, 0 and 1, are available)
  		    time_elapsed); // time in seconds. animation loops if greater than animation time

halo.Draw( vec3f(0,0,0),	// position
	   vec3f(0,0,0),	// rotation
	    0);			// LOD level 

.

Also getting a bone matrix to put a weapon in the hand of the player e.g. is very simple:


int index  = halo.animation.GetBoneIndexOf("joint1"); 
matrix44 m = halo.animation.bones[ index ].matrix;

.

Setting the arm joint individually for shooting a weapon e.g. works as follows:( press F6 in the demo ):


// get the index
int index  = halo.animation.GetBoneIndexOf("joint2"); 
 
// get / modify / set the matrix
matrix44 m = halo.animation.bones[ index ].matrix;
m.x_component()=vec3f(1,0,0);
m.y_component()=vec3f(0,1,0); // set the rotation to identity
m.z_component()=vec3f(0,0,1);
halo.animation.bones[ index ].matrix=m;
 
// re-evaluate the child bones
loopi(0,halo.animation.bones[ index ].childs.size())
{
    halo.animation.EvalSubtree(
        halo.animation.bones[ index ].childs[i], // bone id
        halo.animation.animations[0],            // animation
        -1);                                     // key frame -1 means not use the animation
}

.

Workflow:

  • Design the Model in Maya/MAX/Blender/etc.
  • Export the model using the OgreExporter
  • Convert the model from Ogre binary to Ogre XML (batch file is included)
  • Load the model in the tutorial code

Main Skinning in GLSL:

The main skinning is done in the vertex shader and only requires a few lines.

For the shader, the skinning weights are stored in the color information as 3 floats.

The bone IDs are unpacked from the w coordinate of the position information.

The bone matrixes are stored as simple matrix array.


uniform mat4 bones[100];
uniform int  use_skinning;

void main(void)
{
    mat4 mfinal = gl_ModelViewMatrix ;

    // skinning
    if(use_skinning==1)
    {
    	vec3 weights= gl_Color.xyz;
    	vec3 boneid = gl_Vertex.w * vec3( 1.0/128.0 , 1.0 , 128.0 );
    	boneid = (boneid - floor(boneid))*128.0;

    	mat4 mskin  =	bones[int(boneid.x)]*weights.x+
	    		bones[int(boneid.y)]*weights.y+
			bones[int(boneid.z)]*weights.z;
    	mfinal = mfinal * mskin;
    }

    gl_Position	= gl_ProjectionMatrix * mfinal * vec4(gl_Vertex.xyz,1.0);
}

Note:
Animating Notes for Maya
For Maya, put all animations in one time-line and export them as separate animations.
Ogre Export
Tangents need to be exported as 4D, to include the handedness. The tutorial version does not generate the tangents in the shader as it is faster to read them from the memory.
Bump Mapping
For the Ogre Material file, the bump map needs to be added manually by hand using the texture_bump parameter as follows:
texture Texture\masterchief_base.tif
texture_bump Texture\masterchief_bump_DISPLACEMENT.bmp
Environment Mapping
Environment mapping can be switched on in the material file using the following parameter:
env_map spherical
Clipboard04.png
Advertisement

Today I completed a skinned skeletal animation tutorial, which is very helpful if you are just about to start with game development.

If you just started with game dev you should avoid skeletal animation for a looonnng time smile.png

Nvm, I really appreciate that someone took the time to write an article about this horrible topic wub.png . I just want to point out some stuff, you could consider if you want to update your tutorial in the future.


vec3 weights= gl_Color.xyz;
vec3 boneid = gl_Vertex.w * vec3( 1.0/128.0 , 1.0 , 128.0 );
boneid = (boneid - floor(boneid))*128.0;

mat4 mskin = bones[int(boneid.x)]*weights.x+
bones[int(boneid.y)]*weights.y+
bones[int(boneid.z)]*weights.z;

Some thoughts:

1. I would use custom attributes in times of shaders, no need to reuse all this fixed pipeline stuff, atleast to transport the bone weights. It would be much clearer.

2. Encoding the bone ids in the vertex.w coord is a little bit hacky smile.png A float component has 32bit, so why not use a standard 4x8bit channel. This way it is by far more robust, no magic needed, you get access to 255 bones and it uses the same bandwidth.

3. The same for the weights. Test is out with a 4x8bit channel. You have 4 channels and precision down to 0.5% for skinning, which is enough for me. If you still have issues with the precision, take 4x16bit channel, which is still less bandwidth and supports 4 channels instead of just 3.

Advanced:

4. If you have space for 128 bones, using matrix uniforms, then you could use a combination of a quaternion (4 component) + position (3 component). This way you can use up to 256 bones for the same uniform param space. Btw. you dont need to tranform the quanterion into matrix in the vertex shader, you can rotate the incoming vertex directly by the quaternion.

Ashaman, thank you very much for your detailed reply!


If you just started with game dev you should avoid skeletal animation for a looonnng time smile.png

Well, thats true from the difficulty point of view happy.png

On the other hand, character animation is the most crucial thing that you need right at the beginning to get your game started.

Considering just having character animation and a flat plane with a few boxes (no offense to minecraft here smile.png ) already makes you having a game.

For making the class easy to use also for ppl just getting started with GDev, I have modified the tutorial description on top.

It also includes sample code for drawing a character, getting a bone matrix and setting a bone matrix.


Some thoughts:
1. I would use custom attributes in times of shaders, no need to reuse all this fixed pipeline stuff, atleast to transport the bone weights. It would be much clearer.

Hm.. that would require a major re-write from display-list based to VBO based.

I am not sure it would reduce the number of code-lines, but could be clearer indeed.

I cannot promise to do it anytime soon, but I certainly will post an update in case its done.


2. Encoding the bone ids in the vertex.w coord is a little bit hacky A float component has 32bit, so why not use a standard 4x8bit channel. This way it is by far more robust, no magic needed, you get access to 255 bones and it uses the same bandwidth.

When changing to custom attributes, this will also be done.


3. The same for the weights. Test is out with a 4x8bit channel. You have 4 channels and precision down to 0.5% for skinning, which is enough for me. If you still have issues with the precision, take 4x16bit channel, which is still less bandwidth and supports 4 channels instead of just 3.

Ah, in case somebody would like to change this, thats an easy one:

replace line 180 in mesh.h from

glColor3f(ps.weight[0],ps.weight[1],ps.weight[2]);
to
int w1=ps.weight[0]*255;
int w2=ps.weight[1]*255;
glColor3ub(w1,w2,255-(w1+w2));

Advanced:
4. If you have space for 128 bones, using matrix uniforms, then you could use a combination of a quaternion (4 component) + position (3 component). This way you can use up to 256 bones for the same uniform param space. Btw. you dont need to tranform the quanterion into matrix in the vertex shader, you can rotate the incoming vertex directly by the quaternion.

That sounds advanced indeed. Might not suit for beginners in game development, but sounds like a good improvement when using the class in a commercial project.

Another optimization would be to preprocess the animation tracks. Rrightnow, the entire skeleton is traversed each time. Simply storing the final bone matrices in the animation tracks would already speed up the entire animation.

Further, you could upload the animation tracks as RGBA 16 bit float textures ( in the current version it would be one matrix = 4 pixels). Then,when sampling the texture with linear filtering, you already get the linear interpolated matrix. This would offload the entire animation from cpu to gpu.


you already get the linear interpolated matrix

This is a bad idea, because the matrix interpolation would be unkind to the rotation of bones. Therefor bone rotation is done with slerps (using quaternions). Just keep an eye on the quality of the animation, the vertex shader (where most of the gpu animation stuff is done) isn't the bottleneck in modern game engines ( fragment/pixel shader is often the bottleneck). Btw. using multi-threading to do the bone interpolation plus advanced stuff like blending,overlaying or IK on CPU really works great. Offloading whole animation sequences to the GPU might be useful only in some special case, like massive instancing of animated models etc., but if animation quality is more important, than doing most of the complex stuff on the CPU might be the better and easier solution.

This is a bad idea, because the matrix interpolation would be unkind to the rotation of bones. Therefor bone rotation is done with slerps (using quaternions). Just keep an eye on the quality of the animation, the vertex shader (where most of the gpu animation stuff is done) isn't the bottleneck in modern game engines ( fragment/pixel shader is often the bottleneck). Btw. using multi-threading to do the bone interpolation plus advanced stuff like blending,overlaying or IK on CPU really works great. Offloading whole animation sequences to the GPU might be useful only in some special case, like massive instancing of animated models etc., but if animation quality is more important, than doing most of the complex stuff on the CPU might be the better and easier solution.

Yes, in case of large angles and long interpolation times between two keyframes you might certainly figure out a difference between slerp and linear.

In the tutorial here, the animation is resampled to 20 keyframes per second at the beginning. It takes a bit more memory, but I havent figured out an issue yet when interpolating linearly, since the keyframes are very close to each other.

In case you add complex stuff like rag doll physics etc, CPU is much more flexible of course and shader programming would be a pain in that case.

When benchmarking the current implementation, I found that about half the time from rendering and animation was used for the bone computation on my PC. Maybe pre-calculating the positions/rotations (as written previously) could help, switching to quaternions or also optimizing the math for SSE.

This topic is closed to new replies.

Advertisement