Skin Animation with Vertex Shaders
Author
Can someone explain to me how you would implement skin animation in a vertex shader?
Most tutorials I have been reading on skin animation have suggested either transforming the vertices of the skin mesh manually, placing them in a dynamic buffer for rendering, or to use a vertex shader to perform the entire operation. However, they don''t give code examples or even algorithm layout, and since my vertex shader skills are still very limited, I don''t know where to begin.
Thanks in advance for any response!
There''s four things you need to do:
1) Uploads your bone matrices to vertex shader constants... you''ll want them all in a contiguous block using four constant registers each. Depending on the number of registers available, this will limit the number of bones available to the mesh you are drawing.
2) Add N bone indices and N bone weights to each vertex... N is commonly 4, using one input register for all four indices and another for all four weights.
3) Write VS code that for each bone index:
- looks up the bone matrix from the constant registers
- transforms the position and normal using this bone matrix
- multiplies each result by the corresponding bone weight
- accumulates the weighted results
4) Use these accumulated values in place of the original position and normal.
1) Uploads your bone matrices to vertex shader constants... you''ll want them all in a contiguous block using four constant registers each. Depending on the number of registers available, this will limit the number of bones available to the mesh you are drawing.
2) Add N bone indices and N bone weights to each vertex... N is commonly 4, using one input register for all four indices and another for all four weights.
3) Write VS code that for each bone index:
- looks up the bone matrix from the constant registers
- transforms the position and normal using this bone matrix
- multiplies each result by the corresponding bone weight
- accumulates the weighted results
4) Use these accumulated values in place of the original position and normal.
I'm also interested in skin animation, I have discovered some things, and have some questions too:
so far, common 3d hardware arround has 96 constant registers, so it gives a maximum of 24 bones per model, to me is a very low number. I've read in a nvidia paper that you can upload only 3 columns of the matrix, so you can increase the number up to 32, but I think this is also very limiting.
If you're going to do a pretty simple character you can easily go with 20 bones... but if you add bones to fingers and face, you end up with +50 bones. The nvidia paper say that the only way to go is dividing the model into chunks, based on the bones, and then draw the chunks changing the constant register between drawprimitive calls.
Counting that we are already breaking the models to change renderstates, it is not a big overload to break the model even more?
I've also seen the new MD5 file format it is going to be used in DoomIII , and it seems they're using a variable number of weights per vertex, and I don't think this can be done in vertex shaders.
What I think is that it is better to do the animation/weight vertex calculation in software, and then send the processed vertices to a dynamic buffer, and leave the vertex shader processing for stuff like lightning, I think this is a good balance between cpu/gpu.
Personally, I'm still using the old Dx8 palette skinning in software mode, and I want to switch to hardware... so I would like to know what people think about this before moving to skin shaders.
[edited by - vicviper on July 8, 2003 12:54:19 PM]
so far, common 3d hardware arround has 96 constant registers, so it gives a maximum of 24 bones per model, to me is a very low number. I've read in a nvidia paper that you can upload only 3 columns of the matrix, so you can increase the number up to 32, but I think this is also very limiting.
If you're going to do a pretty simple character you can easily go with 20 bones... but if you add bones to fingers and face, you end up with +50 bones. The nvidia paper say that the only way to go is dividing the model into chunks, based on the bones, and then draw the chunks changing the constant register between drawprimitive calls.
Counting that we are already breaking the models to change renderstates, it is not a big overload to break the model even more?
I've also seen the new MD5 file format it is going to be used in DoomIII , and it seems they're using a variable number of weights per vertex, and I don't think this can be done in vertex shaders.
What I think is that it is better to do the animation/weight vertex calculation in software, and then send the processed vertices to a dynamic buffer, and leave the vertex shader processing for stuff like lightning, I think this is a good balance between cpu/gpu.
Personally, I'm still using the old Dx8 palette skinning in software mode, and I want to switch to hardware... so I would like to know what people think about this before moving to skin shaders.
[edited by - vicviper on July 8, 2003 12:54:19 PM]
We split the model for bones after we''ve split it for materials. Doing transforms in hardware outperforms software transform w/ dynamic buffers many, many times over whether doing 1 bone, or 4 weighted bone transformations. This is the type of work that the vertex shader excels at, and CPUs suck at. Offload the work to the GPU.
Variable bone counts can be done on new hardware with branch/loop support. Old hardware may use a variety of shaders, or just assumes 4 bones and weight the unused bones at 0.
Also, most modelling tools create ''dummy'' bones at the end of each chain that aren''t actually used by any vertex. Removing these saves computation times, reduces the number of matrices that need programming, etc. We have objects that went from over 100 bones to around 80.
Variable bone counts can be done on new hardware with branch/loop support. Old hardware may use a variety of shaders, or just assumes 4 bones and weight the unused bones at 0.
Also, most modelling tools create ''dummy'' bones at the end of each chain that aren''t actually used by any vertex. Removing these saves computation times, reduces the number of matrices that need programming, etc. We have objects that went from over 100 bones to around 80.
Author
Thanks for the info! It helps a lot, I still have one question though...
This may seem really basic, but, how do you know which bone transformation matrix to apply to a given vertex? Are you supplied with a vertex index number when you process a certain vertex that you can compare to some constant values fed to the shader? Also, how do you perform the logic required, because the vertex shader assembly language has limited logic capability.
One more thing. I know that my model''s vertices will only be attached to one bone each, I ampretty sure I will not need to use weight values. So, if I make this assumption in my vertex shader, eliminate weighting code since the weight factor is going to be 1.0 anyway, would there be much of a memory conservation / speed gain?
Thanks again!
This may seem really basic, but, how do you know which bone transformation matrix to apply to a given vertex? Are you supplied with a vertex index number when you process a certain vertex that you can compare to some constant values fed to the shader? Also, how do you perform the logic required, because the vertex shader assembly language has limited logic capability.
One more thing. I know that my model''s vertices will only be attached to one bone each, I ampretty sure I will not need to use weight values. So, if I make this assumption in my vertex shader, eliminate weighting code since the weight factor is going to be 1.0 anyway, would there be much of a memory conservation / speed gain?
Thanks again!
I would like to make a comment about the Doom III reference. I''ve been using and abusing those models and animations for the past several months, and I''d like to point out that they are using a max of 5-6 (not completely sure) bones per vertex, but you must realize that the other vertices HAD weights for ALL the bones when Maya exported the model. Doom III simply optimized the weights == 0.0 out and saved the rest. What you should also remember is that you have a heirarchy for the skeletal structure (usually, or you shouldn''t be using 50+ bones if you''re not!) Therefore, you can break the mesh into smaller subtrees and render only those subtrees (in reality, you might want to render several subtrees with the same constant registers, just to save time). You can, for example, render the right leg and right arm, along with the hip and torso bones, and completely leave the left arma and left leg out, because they have NOTHING to do with the matrices for the right ''limbs''. Of course, you could optimize this for your model, like rendering the entire body in one pass and rendering the head and face (+++ bones!) in another, thus allowing you to use 32 bones in the face, and 32 in the body (or however you want to arrange it). Good luck!
Chris Pergrossi
< ctoan >
My Realm
Chris Pergrossi
< ctoan >
My Realm
In response to your question of ''How to know the final transformation matrix in the vertex shader'' (something to that effect 
), I''ll tell you what we did for our engine. Instead of saving the current transformation matrix for each bone when you update the skeleton (in animation or bind position, you can do both), simply concate the bones current transformation with that of ALL it''s parents. That way, in the vertex shader, you can simply multiply by the vertex''s bone''s rotation matrix and in actuality get all the transformations from all it''s parents up the tree. I don''t know how to reference the constant registers with a variable though, because we use Cg (C for Graphics) and it generates the actual vertex shader at run-time, so *shrug*. But I tell you, it''s MUCH MUCH MUCH faster doing skeletal animation with vertex shaders than with software. You can see this if you write a animation viewer in software, view the profile view (see how long it''s taking) then shift to 3D Mark 2001 SE and take a look at the Vertex Shading test... :O Hehehe, again, good luck! I was working on a similar problem for a month to no avail, and then had a lighting strike at about 2 in the morning while playing Resident Evil Zero (coooooool!), so always keep a brain process on the topic!
Chris Pergrossi
< ctoan >
My Realm
), I''ll tell you what we did for our engine. Instead of saving the current transformation matrix for each bone when you update the skeleton (in animation or bind position, you can do both), simply concate the bones current transformation with that of ALL it''s parents. That way, in the vertex shader, you can simply multiply by the vertex''s bone''s rotation matrix and in actuality get all the transformations from all it''s parents up the tree. I don''t know how to reference the constant registers with a variable though, because we use Cg (C for Graphics) and it generates the actual vertex shader at run-time, so *shrug*. But I tell you, it''s MUCH MUCH MUCH faster doing skeletal animation with vertex shaders than with software. You can see this if you write a animation viewer in software, view the profile view (see how long it''s taking) then shift to 3D Mark 2001 SE and take a look at the Vertex Shading test... :O Hehehe, again, good luck! I was working on a similar problem for a month to no avail, and then had a lighting strike at about 2 in the morning while playing Resident Evil Zero (coooooool!), so always keep a brain process on the topic! Chris Pergrossi
< ctoan >
My Realm
Alex: if you''re not going to use weighted vertices, you can speedup/simplify the code a lot.
Just do this: for every vertex, you will have an index to the bone the vertex is attached, so when you compute that vertex, you apply the matrix of the indexed bone. Usually, the index to the bone is provided by the 3d edition tools you are using.
Anyway, if you use vertex shaders, you will still be limited by the limited memory for constant registers, so the max number of bones of 24/32 still apply.
About DoomIII, I know that when I export, they get a huge list of bone indices/weights per vertex, I had that problem myself when exporting physique data, because I had to cap the indices to 4 or 2.... Anyway, I''m still insterested on see how they handled animation characters in Doom 3.... is there any source code avaliable for viewing doom 3 models? (using vshaders, already have non vshader samples)
Author
Just a quick question about bone weights: What is the use of attaching vertices to more than one bone? What animation effects can it give you?
multiple bones per vertex are commonly used to avoid a problem with vertices using a single bone.
The problem is that if you're going to animate a human or any other organic character, sometimes you want the vertices of the joint to blend smoothly instead of go with the bone all the way... just put yourself in the case of a tentacle or the tail of a dinosaur.
the bad thing is weighted vertices are computationally expensive.
[edited by - vicviper on July 9, 2003 11:19:41 AM]
The problem is that if you're going to animate a human or any other organic character, sometimes you want the vertices of the joint to blend smoothly instead of go with the bone all the way... just put yourself in the case of a tentacle or the tail of a dinosaur.
the bad thing is weighted vertices are computationally expensive.
[edited by - vicviper on July 9, 2003 11:19:41 AM]
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
