Hello
I have read the Frank Luna paper on "Skinned Mesh Character Animation with Direct3D 9.0c", some tutorials and a few topics on this forum. But I am still confused about the relationship between meshcontainers and frames(bones). I know the "theory" but don't understand exactly how it is implemented in DX9.
It seems that the vertex format is not loaded as a skinned mesh. Is this specific for X. files?
A model mesh or skin is copied in to a meshcontainer, and the meshcontainer is pointed to by a frame. Does that mean that you can not have more meshcontainers then bones? Does it matter which bone is pointing to which meshcontainer? Or is it just a way of "linking" them? If I can have a model split up in several meshcontainers, when do I want to do this? Will this "disconnect/break up" and not make a continuous skin/mesh? For an RPG game, would you have the "body armor piece" connected to the body bone etc?
Why is the implementation this way, why not just load everything from file directly in to an object that has everything ready for drawing?
Since you have an palette of transform matrices, each vertex has indices pointing to the ones that affect it and a weight to each one, right?
Is there some visual representation of how these things fit together? It is hard for me to grasp where things are during the loading process and during run-time.
Any comment appreciated.
[DX9] Implementation of Meshcontainers and Frames
Hello
I have read the Frank Luna paper on "Skinned Mesh Character Animation with Direct3D 9.0c", some tutorials and a few topics on this forum. But I am still confused about the relationship between meshcontainers and frames(bones). I know the "theory" but don't understand exactly how it is implemented in DX9.
It seems that the vertex format is not loaded as a skinned mesh. Is this specific for X. files?
A model mesh or skin is copied in to a meshcontainer, and the meshcontainer is pointed to by a frame. Does that mean that you can not have more meshcontainers then bones? Does it matter which bone is pointing to which meshcontainer? Or is it just a way of "linking" them? If I can have a model split up in several meshcontainers, when do I want to do this? Will this "disconnect/break up" and not make a continuous skin/mesh? For an RPG game, would you have the "body armor piece" connected to the body bone etc?
Why is the implementation this way, why not just load everything from file directly in to an object that has everything ready for drawing?
Since you have an palette of transform matrices, each vertex has indices pointing to the ones that affect it and a weight to each one, right?
Is there some visual representation of how these things fit together? It is hard for me to grasp where things are during the loading process and during run-time.
Any comment appreciated.
The best way to think about this is that the way skinned mesh works is that. You have n Bones where each Bone contains Position/Rotation or any additional info you need. Then you have Frames where each Frame contains information for each Bones. So if you had 10 bones in the model and you have to 2 frames then you would just take the number of bones*number of frames to figure out how long the animation will be. Then you have the MeshContainer which contains an handle to the Frames. One of the problems with D3DXFRAME is that it store Bone Transformation which is a really bad design because it makes it almost impossible to have two of the same models with the same vertexbuffer using an different animation. Here is an example of how i have everything set up in my engine.
struct Bone
{
GLGXVECTOR3 position;
GLGXQUATERNION rotation;
};
struct Frame
{
Bone* bones;
}
class MeshContainer
{
Frame* frames;
};
The vertices holds the vertex weight and bone index which they are been influence by.
Author
Frames as in bones or frames as in render a frame?
What? animation length depends on what timeline the animation keys are at and how fast I am playing the track.
I know the theory, but confused about the DX9 implementation. Do most games use the DX animationController or do they have a custom made one?
A frame struct is the bone in my understanding.
Can you MSPAINT me a quick overview of how frames, meshcontainers, animcontroller etc fit together during loading and run-time?
Thanks for the comments, keep em coming
So if you had 10 bones in the model and you have to 2 frames then you would just take the number of bones*number of frames to figure out how long the animation will be.
What? animation length depends on what timeline the animation keys are at and how fast I am playing the track.
I know the theory, but confused about the DX9 implementation. Do most games use the DX animationController or do they have a custom made one?
A frame struct is the bone in my understanding.
Can you MSPAINT me a quick overview of how frames, meshcontainers, animcontroller etc fit together during loading and run-time?
Thanks for the comments, keep em coming
Let me take a crack at explaining this to you, though I think Frank Luna's paper says it clearer ( I know you've read it already ).
Question 1:
It seems that the vertex format is not loaded as a skinned mesh. Is this specific for X. files?
Answer 1: I assume when you say vertex format you mean vertex declaration and when you say it is not loaded as a "skinned mesh" you mean the vertex declaration does not have blend weights and blend indices in it after you load an X file using one of the D3DX APIs. I can't answer if this is specific only to .X files because there are tons of different file formats out there. Let's just assume the answer is "yes", if you load an X file using D3DXLoadMeshFromHierarchy() then since you are required to provide the overloaded implementation of the loading classes, the vertex format is not correctly expanded to include "skinning data" and you must do it yourself as shown in Luna's examples.
Question 2:
A model mesh or skin is copied in to a meshcontainer, and the meshcontainer is pointed to by a frame. Does that mean that you can not have more meshcontainers then bones?
Answer 2:
You can have more meshcontainers than bones. There is a field in the MeshContainer structure called "Next" which allows you to form an arbitarily long linked list of MeshContainers underneath any bone (frame) that you wish.
Question 3:
Does it matter which bone is pointing to which meshcontainer? Or is it just a way of "linking" them?
Answer 3:
The skinned mesh is represented as a frame hierarchy. In a hierarchy, being above, below or at the same level matters. In a frame hierarchy, if you attach a meshcontainer underneath a particular frame, the meshcontainer's vertex positions will be transformed by all the frame transformations above it (all the way up to the root). This is how weapons and shields are attached to skinned characters in games. The meshcontainer for a weapon is linked underneath the "hand" bone. Because of that the weapon is drawn at the current position of the "hand" bone each frame.
Question 4:
If I can have a model split up in several meshcontainers, when do I want to do this? Will this "disconnect/break up" and not make a continuous skin/mesh? For an RPG game, would you have the "body armor piece" connected to the body bone etc?
Answer 4:
You can have a model split up into several meshcontainers, real games do this all the time for player body part customization. The meshcontainers won't be continuous, but if your artists model them so they overlap a bit, nobody will notice. I think you are confusing the term "connected" with two different things here. The first term I like to call "attached", like when you link a meshcontainer underneath a bone in the hierarchy. A weapon meshcontainer could be attached to a hand bone, this does not mean that the weapon vertices are influenced by the hand bone during the vertex skinning however. It just uniformly transforms *ALL* the weapon vertices by the "toRoot" transform of the hand bone. So the second term is called "influenced". When a meshcontainer is influenced by a bone it means the bone appears in the blend indices list for one or more vertices of this meshcontainer's vertex buffer. It means the final world space position of that bone will deform one or more of the vertices of this meshcontainer. In your question above, the "body armor piece" should be influenced by the bones that make up the body (like the spine, pelvis, hips, clavicles, etc). The body armor piece would not be attached to those bones, it would be attached under the "Scene Root" bone most likely.
Question 5:
Why is the implementation this way, why not just load everything from file directly in to an object that has everything ready for drawing?
Answer 5:
I assume you mean the D3DX meshcontainer/frame and .X file terminology? The answer is really anybody's guess. As I said, most people don't use the D3DX allocation hierarchy or the ID3DXAnimationController in real products and most real engines can load everything into a scene graph that is ready for rendering. You still need to load your data hierarchically or else forward kinematics doesn't work.
Question 6:
Since you have an palette of transform matrices, each vertex has indices pointing to the ones that affect it and a weight to each one, right?
Answer 6:
There are two sides to the matrix palette in both Luna's and the DX9 examples. On the CPU side the matrix palette is just an array of pointers to the TransformationMatrix from particular frames of the skinned mesh hierarchy. In this way when the CPU hierarchically updates the TransformationMatrix field of each frame in the recursive step, the matrix palette is automatically up-to-date. Each frame the matrix palette is uploaded to the GPU. On the GPU side, the vertices coming into the vertex shader do have up to 4 integer indices which point into the matrix palette as it sits in the constants registers of the GPU. As you stated each vertex also has an array of up to 4 weights (which must sum to 1.0f (100%)). Inside the vertex shader the incoming local vertex position is adjusted via a weighted averaging (as shown in Luna's paper). After this the vertex position (and normal) can be transformed by world, view, projection matrices to enter clip space.
Question 7:
Is there some visual representation of how these things fit together? It is hard for me to grasp where things are during the loading process and during run-time.
Answer 7:
I don't have time to draw such a diagram, but Luna actually has included a better diagram than his online paper has in his book, "Intro to 3d game programming in dx9 - a shader approach".
Question 1:
It seems that the vertex format is not loaded as a skinned mesh. Is this specific for X. files?
Answer 1: I assume when you say vertex format you mean vertex declaration and when you say it is not loaded as a "skinned mesh" you mean the vertex declaration does not have blend weights and blend indices in it after you load an X file using one of the D3DX APIs. I can't answer if this is specific only to .X files because there are tons of different file formats out there. Let's just assume the answer is "yes", if you load an X file using D3DXLoadMeshFromHierarchy() then since you are required to provide the overloaded implementation of the loading classes, the vertex format is not correctly expanded to include "skinning data" and you must do it yourself as shown in Luna's examples.
Question 2:
A model mesh or skin is copied in to a meshcontainer, and the meshcontainer is pointed to by a frame. Does that mean that you can not have more meshcontainers then bones?
Answer 2:
You can have more meshcontainers than bones. There is a field in the MeshContainer structure called "Next" which allows you to form an arbitarily long linked list of MeshContainers underneath any bone (frame) that you wish.
Question 3:
Does it matter which bone is pointing to which meshcontainer? Or is it just a way of "linking" them?
Answer 3:
The skinned mesh is represented as a frame hierarchy. In a hierarchy, being above, below or at the same level matters. In a frame hierarchy, if you attach a meshcontainer underneath a particular frame, the meshcontainer's vertex positions will be transformed by all the frame transformations above it (all the way up to the root). This is how weapons and shields are attached to skinned characters in games. The meshcontainer for a weapon is linked underneath the "hand" bone. Because of that the weapon is drawn at the current position of the "hand" bone each frame.
Question 4:
If I can have a model split up in several meshcontainers, when do I want to do this? Will this "disconnect/break up" and not make a continuous skin/mesh? For an RPG game, would you have the "body armor piece" connected to the body bone etc?
Answer 4:
You can have a model split up into several meshcontainers, real games do this all the time for player body part customization. The meshcontainers won't be continuous, but if your artists model them so they overlap a bit, nobody will notice. I think you are confusing the term "connected" with two different things here. The first term I like to call "attached", like when you link a meshcontainer underneath a bone in the hierarchy. A weapon meshcontainer could be attached to a hand bone, this does not mean that the weapon vertices are influenced by the hand bone during the vertex skinning however. It just uniformly transforms *ALL* the weapon vertices by the "toRoot" transform of the hand bone. So the second term is called "influenced". When a meshcontainer is influenced by a bone it means the bone appears in the blend indices list for one or more vertices of this meshcontainer's vertex buffer. It means the final world space position of that bone will deform one or more of the vertices of this meshcontainer. In your question above, the "body armor piece" should be influenced by the bones that make up the body (like the spine, pelvis, hips, clavicles, etc). The body armor piece would not be attached to those bones, it would be attached under the "Scene Root" bone most likely.
Question 5:
Why is the implementation this way, why not just load everything from file directly in to an object that has everything ready for drawing?
Answer 5:
I assume you mean the D3DX meshcontainer/frame and .X file terminology? The answer is really anybody's guess. As I said, most people don't use the D3DX allocation hierarchy or the ID3DXAnimationController in real products and most real engines can load everything into a scene graph that is ready for rendering. You still need to load your data hierarchically or else forward kinematics doesn't work.
Question 6:
Since you have an palette of transform matrices, each vertex has indices pointing to the ones that affect it and a weight to each one, right?
Answer 6:
There are two sides to the matrix palette in both Luna's and the DX9 examples. On the CPU side the matrix palette is just an array of pointers to the TransformationMatrix from particular frames of the skinned mesh hierarchy. In this way when the CPU hierarchically updates the TransformationMatrix field of each frame in the recursive step, the matrix palette is automatically up-to-date. Each frame the matrix palette is uploaded to the GPU. On the GPU side, the vertices coming into the vertex shader do have up to 4 integer indices which point into the matrix palette as it sits in the constants registers of the GPU. As you stated each vertex also has an array of up to 4 weights (which must sum to 1.0f (100%)). Inside the vertex shader the incoming local vertex position is adjusted via a weighted averaging (as shown in Luna's paper). After this the vertex position (and normal) can be transformed by world, view, projection matrices to enter clip space.
Question 7:
Is there some visual representation of how these things fit together? It is hard for me to grasp where things are during the loading process and during run-time.
Answer 7:
I don't have time to draw such a diagram, but Luna actually has included a better diagram than his online paper has in his book, "Intro to 3d game programming in dx9 - a shader approach".
Author
Thx for those awnsers.
One thing I still don't undertand; after D3DLoadMeshHierarcyFromX() is called I am supposed to "link frames" to their mesh containers, i.e populate the matrix palette with pointers. However it seems to me that after D3DLoadMeshHierarcyFromX() the frames are only initialized and not populated with any mesh container pointers, i.e D3DXFRAME.pMeshContainer is still blank. How can then the tutorial check pFrame->pMeshContainer during the linking step? What am I missing?
Again, thx.
One thing I still don't undertand; after D3DLoadMeshHierarcyFromX() is called I am supposed to "link frames" to their mesh containers, i.e populate the matrix palette with pointers. However it seems to me that after D3DLoadMeshHierarcyFromX() the frames are only initialized and not populated with any mesh container pointers, i.e D3DXFRAME.pMeshContainer is still blank. How can then the tutorial check pFrame->pMeshContainer during the linking step? What am I missing?
Again, thx.
Thx for those awnsers.
One thing I still don't undertand; after D3DLoadMeshHierarcyFromX() is called I am supposed to "link frames" to their mesh containers, i.e populate the matrix palette with pointers. However it seems to me that after D3DLoadMeshHierarcyFromX() the frames are only initialized and not populated with any mesh container pointers, i.e D3DXFRAME.pMeshContainer is still blank. How can then the tutorial check pFrame->pMeshContainer during the linking step? What am I missing?
Again, thx.
Hi Tispe,
Can you please post the code snippet that is confusing you? You have two creation methods you must implement for an ID3DXAllocateHierarchy, CreateFrame and CreateMeshContainer. D3DX will link mesh containers to frames after the D3DXLoadMeshHierarchyFromX() API completes provided you returned non NULL frames and mesh containers at each of these creation methods.
Author
Hi again
I didn't think about D3DXLoadMeshHierarchyFromX() beeing able to initialize pFrame->pMeshContainer, sibling and child on its own after the allocation functions has been run. I guess that is whats going on.
So "attached" MeshContainers are recognized via the D3DXFRAME:pMeshContainer, and "influenced" MeshContainers are recognized via the matrix pallete.
If I ATTACH my gun to the hand bone, it gets translated to where the hand is, and if Boots are attached to the feet they get translated to where the feet bones are.
Lets say a game where the character can switch weapon, hat, jacket, gloves, pants and boots they all get attached to their respective bones. But the glove meshcontainer is only influenced by fingers, hand and forearm bones.
MS Tiny.x has just one mesh, meaning one meshcontainer. Probably linked to the root frame. And that MeshContainer is influanced by all bones.
It is kinda hard to grasp without an illustration. Lets say I want earmuffs and a hat, I would need two meshcontainers for each both attached to the head, meaning I have to chain them D3DXMESHCONTAINER* pNextMeshContainer.
I didn't think about D3DXLoadMeshHierarchyFromX() beeing able to initialize pFrame->pMeshContainer, sibling and child on its own after the allocation functions has been run. I guess that is whats going on.
So "attached" MeshContainers are recognized via the D3DXFRAME:pMeshContainer, and "influenced" MeshContainers are recognized via the matrix pallete.
If I ATTACH my gun to the hand bone, it gets translated to where the hand is, and if Boots are attached to the feet they get translated to where the feet bones are.
Lets say a game where the character can switch weapon, hat, jacket, gloves, pants and boots they all get attached to their respective bones. But the glove meshcontainer is only influenced by fingers, hand and forearm bones.
MS Tiny.x has just one mesh, meaning one meshcontainer. Probably linked to the root frame. And that MeshContainer is influanced by all bones.
It is kinda hard to grasp without an illustration. Lets say I want earmuffs and a hat, I would need two meshcontainers for each both attached to the head, meaning I have to chain them D3DXMESHCONTAINER* pNextMeshContainer.
Author
OK, my second attempt to figure out how this all fit together.
I know atleast how the hand vertices are blended (by using up to four matrices of the pallete).
What I don't quite get is how a hand mesh is translated to the hand bone. Obviously I need to multiply all the hand vertices by the hand CombTransformationMatrix, but the four bone indices of a vertex is supposed to add up to 1.0. Do I need to use the first of the four bone indices and do a 100% multiplication to translate the hand, then use the remaining three indices and multiply by up to three bones that also add up to 100%? This would mean a 200% multiplication. I figure the Offset Matrix is for another purpose, to translate from bone space to model space.
Please enlighten me about when the hand vertices are first transformed to the correct frame position.
As demonstrated in the below picture, when does the hat acutally get placed ontop of the head. Do I use the first BoneIndex for a 100% multiplication?
I know atleast how the hand vertices are blended (by using up to four matrices of the pallete).
What I don't quite get is how a hand mesh is translated to the hand bone. Obviously I need to multiply all the hand vertices by the hand CombTransformationMatrix, but the four bone indices of a vertex is supposed to add up to 1.0. Do I need to use the first of the four bone indices and do a 100% multiplication to translate the hand, then use the remaining three indices and multiply by up to three bones that also add up to 100%? This would mean a 200% multiplication. I figure the Offset Matrix is for another purpose, to translate from bone space to model space.
Please enlighten me about when the hand vertices are first transformed to the correct frame position.
As demonstrated in the below picture, when does the hat acutally get placed ontop of the head. Do I use the first BoneIndex for a 100% multiplication?
OK, my second attempt to figure out how this all fit together.
I know atleast how the hand vertices are blended (by using up to four matrices of the pallete).
What I don't quite get is how a hand mesh is translated to the hand bone. Obviously I need to multiply all the hand vertices by the hand CombTransformationMatrix, but the four bone indices of a vertex is supposed to add up to 1.0. Do I need to use the first of the four bone indices and do a 100% multiplication to translate the hand, then use the remaining three indices and multiply by up to three bones that also add up to 100%? This would mean a 200% multiplication. I figure the Offset Matrix is for another purpose, to translate from bone space to model space.
Please enlighten me about when the hand vertices are first transformed to the correct frame position.
As demonstrated in the below picture, when does the hat acutally get placed ontop of the head. Do I use the first BoneIndex for a 100% multiplication?
This really depends on how your code traverses the frame hierarchy during the render loop. If it traverses it from top (Head) to bottom, then the Hat will be drawn first. You do not however show the contents of the meshcontainer's vertex buffer which is where the bone indices and weights are stored (1 per vertex). So I cannot guess how the blending happens.
I definitely think you should re-read Luna's paper about blending you are confused by how the weighted averaging of a vertex position occurs. First you transform all the frame matrices in your hierarchy using the current time and a set of keyframes. Then you render each meshcontainer as you encounter it using your traversal code in your render loop (usually top-down recursive). For each vertex of a meshcontainer, you take the 4x4 bone transform matrix referenced by that vertices bone index and transform the position by it and multiply by the bone weight. Then you add that to the next weighted position. Do this up to four times and you have the final Object space position of that vertex. Now multiply it by World, View, Projection matrix and its in clip space.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
