Skeletal Animation: Rotations per Vertex

Recommended Posts

dachande    162
Hey all, I've had a look around (in the wrong places, I suspect) and have failed to find the answers I'm looking for. None of the resources I've found have been quite clear enough! Perhaps somebody could clear something up for me? I'm looking in to skeletal animation - and know that I am required to rotate each vertex in a model by the appropriate bone(s) rotation. Should those rotations be performed on each vertex, just before they are sent down to be rendered? Or should I be performing one rotation for each bone on the modelview matrix, and send down all vertices for that bone with no further calculations? The problem with the latter method, of course, is ensuring that the triangles that connect vertices between one bone and another will remain connected. Lastly, I've been unable to appertain from my resources, whether or not I would be able to use a vertex shader to rotate each vertex accordingly - are vertex shaders capable of rotating vertices? (Apologies, I'm quite new to shaders...) Any hints on the matter would be much appreciated! Thanks in advance, Dachande

Share on other sites
Supernat02    604
Here are my thoughts. Keep track of the matrices in a matrix stack. You can use the one included with DirectX or create your own. When you render the mesh, apply that bone's transformation and then draw that bone. Of course, each frame you have to apply the bone transformations in a parent-child order (i.e. you apply the top transformation down to each child, etc.). This will ensure that vertices remain intact as long as you are only applying rotations...I think...

Chris

Share on other sites
You can't and shouldn't send the data down for one bone at a time. First, the bones are assigned to vertices, not faces. A face may have vertices affected by different bones. In fact, most bone implementations allow each vertex to be affected by 4 bones, so it's possible (though infinitely unlikely) that a single face is affected by 12 different bones.

Your options, and you've got them right, are software transforming, and vertex shaders.

Software pros:
Will work on all hardware.
Can use any number of influences per vertex (though needing very many is unlikely)
If you're doing many render passes, you pay for the transform once.
No need to split mesh up into pieces (see shaders)
Can take advantage of HW T&L on GeForce, GeForce 2, and GeForce 4MX cards.

Software cons:
Can be slow, CPU is often busy doing other things like AI, Networking, Graphics preparation, mixing audio, etc.
Needs many paths to be optimal (SSE, 3DNow, etc).

Parallel processing with CPU, so it's fast.

Needs newer hardware for hardware shader processing
Older hardware can software emulate shader, but then you don't take advantage of HW T&L of low end GeForce cards.
Requires a pre-processing step to split the mesh into pieces. vs.1.1 has 96 constants, some of which you'll use for materials, lights, fog, texture transforms, etc. If you're lucky you'll fit 20 bone matrices in. If you have a mesh with many bones, you'll need to break it up into sections that use up to 20 bones.
Work is done multiple times when doing multi-pass rendering. This is a non-issue if the polys are large enough because you'll be fill bound, not transform bound. Then again, on older cards that don't do hardware shaders, it's lots of work redone on the CPU.

If you require a card with hardware shaders, they're a definate win. On HW T&L but non-hardware shader cards, they're a possible loss, possible win. If you do multipass they're probably a loss. On very old card with neither HW T&L or shaders, software emulated shaders probably comes to about the same speed as your own software transform. If you do multipass on one of these old cards, your own software transform becomes a win.

And finally, if you're not CPU limited, and have a fast CPU (like me with a P4 2.8 and a GeForce3), using software processing may be a win since your CPU, which would have been idle anyway, can take load off your GPU.

In other words, it's damn complex.

Share on other sites
vajuras    139
Additonal resource: Skeletal Animation Tutorial

Like NameThatNobodyElseTook says, shaders are indeed faster and the way to go. Using VS 3_0, it's possible to skin w/o resorting to splitting apart the mesh. I have a friend that did this at his job. Not sure how it performed but i imagine it's probably pretty fast. (Btw, if anyone has an example of a texture lookup in VS 3_0 please share)

Myself I am currently working on using VS 1_1 and splitting apart the mesh. I searched this forum, only saw a post by namethatnobodyelse that described the logic behind it. Which I've been trying to get to work. hopefully I'll have it wrapped up today or tomorrow (splitting apart meshes that have over 20 bones).

Share on other sites
dachande    162
Thanks guys, that's great. I think that's the info I need to get me well on my way.

My plan at this point then, is to create both a software and a VS version of skeletal animation, so that I may select the most appropriate at the time.

Could someone quickly run by me the idea of having to split a mesh when using vertex shaders (i.e. further detail as to why)? Like I explained above, I'm very new to shaders, but if I can find a few why's and how's about them, I'm sure I'll pick them up soon enough ;)

Thanks again,

Dachande

Share on other sites
There are many different ways to split the mesh, so I won't cover how. I have a technique I came up with and it seems to do a good job. Perhaps it's not optimal, but it works. There's a post somewhere here on gamedev that you can search for it you really want something like my technique.

As to the why you need it, consider this:

We have a character with 60 bones. Each bone requires a matrix to be programmed into your shader's constants space. You don't need the 4th column (row after transpose) of the matrix, so we can store what we need in 3 constant registers. If you want to do the proper normal calculation, it needs the inverse-transpose of the world transform, so you'll need 6 constant registers per bone. I use 3, which means I can't do shears or scale each axis differently and expect lighting to work.

In vs.1.1 (GeForce 3, GeForce 4, Radeon 8x00), you have 96 constants registers. You can decided what calculations you want, but typically you'll need some constants for fog, possibly some for texture transformation, definately some for lights (2-3 regs per light). Ambient and emissive material properties perhaps. Specular power. View, Projection matrices. Some generic constants such as 0, 1, 0.5, 765.1. I use 765.1 with the boneids stored in a D3DCOLOR... it's 255*3 + a bit because of floating point rounding. Use whatever value helps you get to the appropriate bone constant register. You could store the boneids are 4 floats, and not need to multiply, but I prefer to save space.

Ok, so we have all these other constants we need to worry about. Lets assume you have 60 constants remaining. Each bone taking 3 or 6 constants means you can hold 20, or 10 bone transforms at a time. If we break the mesh into pieces that each use <= 20 bones, and program whichever bones are necessary for each section before we render, we can now render a mesh of any number of bones, instead of hitting a hard limit of 20 bones total for our mesh.

For each section you need to know the active set of bones. char *m_aActiveBones, int m_nActiveBones. In each vertex you need to put the index into the active bones. If a vertex used bone 5, and the active bone list array was {0, 2, 5, 6, 9, 12}, in the vertex we'd put 2 (m_aActiveBones[2]==5). When you program the constants, you loop from 0 to m_nActiveBones, and program m_aBoneMatrix[m_aActiveBones[i]]. The the vertex shader, you take the bone id (in our example, 2, which will be 2/255 if you use D3DCOLOR), multiply by 765.1 = 6.0008. Then move that into A0.x, and read your matrix table c[a0.x + base]. The result is that you use bone matrix 5 on that vertex, which happens to be the 3rd matrix in your constants.

Share on other sites
dachande    162
Hey,

So as I understand it the splitting suggests that the vertex shader is asked to do all the work for an animation - is this a correct assumption?

Would it be feasible to avoid splitting a mesh if the shader technique were to be combined with some software processing. I.E - calculate the the bone matrices as per the software method, and send to the shader the initial matrix and an offset matrix and ask the GPU to just perform the interpolation work?

This would require setting shader constants approximately once for each bone in a model - my lack of familiarity with shaders lets me down here - I'm as of yet unsure if settings constants multiple time causes a performance hit, or if it's normal to send constants down with a vertex. I'm assuming that setting constants should be kept to a minimal?

Does this sound like something that might work, or am I totally off track here?

Thanks!,

Dachande

Share on other sites
The above was assuming the CPU did all the animation work. Interpolating keyframes, mixing animation, building the matrices.

The GPU is just applying 1-4 matrices to each vertex. The limited storage for bone matrices is the reason we need to split. We can fit only 20 matrices at a time, so if you have more than 20 bones, you have more than 20 matrices, and they don't all fit in vs.1.1 constant space at once.

Setting the constants per vertex (ie: Just more vertex data) would require changing the vertices whenever a bone moves... you might as well do software skinning.

You set constants before calling DrawIndexedPrimitive. Each call has quite a bit of overhead... best to set all constants in 1 SetVertexShaderContant() call per Draw()

Share on other sites
dachande    162
I *think* I'm beginning to understand - does that mean that for all the 20 matrices that have been set as constants in the vertex shader, that only 3-4 of those matrices will actually be used on any one vertex?

Setting all the matrices at the beginning of the render speeds things up, as setting constants before drawing each bone is too slow.

Apologies if this is what you've been trying to explain for the last couple of messages ;)

Am I close?

Thanks for your patience,

Dachande

Share on other sites
vajuras    139
hey i just got nobodyelsename's method to work this morning (took a few hrs). Wasn't really splitting apart a mesh per se- just build a new VB/IB for each 20 bones. then mix this with matrix palette skinning (see NVSDK 8 PaletteSkin project) and it should work just fine. I'll probably post the code on my site once i get the lighting to look better. it's basically the exact same process as my software skinning tutorial. just that instead of letting the bone transform the vertices- this takes place on the GPU.

Share on other sites
Quote:
 Original post by dachandeI *think* I'm beginning to understand - does that mean that for all the 20 matrices that have been set as constants in the vertex shader, that only 3-4 of those matrices will actually be used on any one vertex?

Yes.
Quote:
 Setting all the matrices at the beginning of the render speeds things up, as setting constants before drawing each bone is too slow.

No.

It's not just an optimization, it's needed. First lets consider a simple case.

1---2|  /|| / ||/  |3---4

Lets say vertex 1 uses bone 1, and vertex 2-4 use bone 2. How can you render that one bone at a time? You can't. You have to render full faces.

1--2,5--7|  /|  /|| / | / ||/  |/  |3--4,6--8

The middle vertices are split such that we can assign different bones (this isn't really done, this is just trying to make the faces use individual bones).

Now lets say vertices 1-4 use bone 1, and vertices 5-8 use bone 2. Now lets translate bone 2 to the side.

1---2  5---7|  /|  |  /|| / |  | / ||/  |  |/  |3---4  6---8

Hmm. We've got a hole in the mesh. Unless different vertices of the same face use different bones, you cannot avoid holes. If a face uses more than 1 bone, then we can't possibly render one bone at a time.

Ok, so then how do you render them?
x..x.. . . .. a..x|\ .| \.b--c---d...x\  |  /.  .. \ | / . . .  \|/  ..  .   e...x...x

Imagine this L shaped "elbow" joint... I've greyed out some vertices and faces to show the shape. The bone assignments here would be as this:

(single influence)
a,b,c = bone1
d,e = bone2

In this case, all three polys around the joint need 2 bones. Using a single bone per vertex can produce ugly artifacts though. Great for metal objects, like robot arms.

(2 influences)
a = bone1
b = bone1 70%, bone 2 30%
c = bone1 50%, bone 2 50%
d = bone2
e = bone1 30%, bone 2 70%

In this case, not only do the three polys require 2 bones, but most of the vertices require 2 bones. Extra influence, and gradually changing influences can produce nice curved joints. Great for organic fleshy objects that have a curve to their joints.

Share on other sites
dachande    162
Thanks, that's what I needed to know. To be honest, I'd already figured out the majority of the stuff in the last post - but it's very helpful to have someone confirm that you're on the right tracks. (Thanks for putting in the effort to draw me some ascii pictures too! :) )

Vajuras, I quite like the tutorial you've posted on your site - I look forward to seeing the shader version, it's always a good idea to comapare notes with with something that's already been implemented. I've been unable to run your GODZ download though - it crashes as soon as it's clicked ;)

I've done a quick search for the post describing Namethatnobodyelsetook's method for splitting meshes so that I can reference it here for anyone else finding this post useful. There are a few hundred posts to dig through though! I assume Vajuras knows the link? Could you post it if you get a sec? ;)

Thanks,

Dachande

Share on other sites
vajuras    139
here ya go: skeletal animation in Direct3D

Never seen the engine crash unless dx9.0c (summer 2004 release) wasn;t installed. make sure you have that. Btw, feel free to post what the log says. That way I know 4 sure where it crashes.

[Edited by - vajuras on October 4, 2004 9:32:20 AM]

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account