Jump to content
  • Advertisement
B. /

DX11 GPU Skinning Problem

Recommended Posts

1 hour ago, B. / said:

which Format would be the best for GPU Skinning, maybe FBX?

Seems better, yes. But i'm totally no expert with file formats. There are many shipped games using collada - i would not drop this if you already invested some time. Maybe the missing data is stored elsewhere in the file? Maybe open a new collada related topic so people knowing better can help. (Say also which modelng app you use then.)

1 hour ago, B. / said:

And for CPU Skinning i would need a origin Vertex List and transform these and set it to the vertexbuffer for every fps?

Yes, but performance does not matter - just to proof you have correct data and your algorithm is right if you can't get it to work otherwise. (I have a debug visualization class that buffers lines, point or polygons with given color and drwas once per frame. This would do and is very useful for pretty much anything.)

Share this post


Link to post
Share on other sites
Advertisement

Hi Joel,

I found my mistake, the 12 vertex weights was linked with the 12 vertices pos array of the file and now i get the same result as assimp :)

So now i load the weigths and Bone Indices right and if i only send Matrix Identitys to the shader, the model will draw correct.

Send I the origin bone matrices, that i load from the file, I get a Science-Fiction effect by drawing the model. See image

So the last thing to do is now to send the right transformed matrices to the shader.

Here the code how i send these to the shader (The BindPose Matrix is invert and the result is a Matrix Idetity)

 

                List<Matrix> boneMatrices = new List<Matrix>();
                boneMatrices.Add(Matrix.Identity);

                foreach (Joint joint in this.bones)
                {
                    Matrix m = MatrixHelper.CalculateMatrixFromParents(joint, Matrix.Identity);
                    boneMatrices.Add(m * this.bindPoseMatrix);
                }

                return boneMatrices;

 

Anyone an idea how to solve the last stept?

Greets

Benjamin

03.jpg

Share this post


Link to post
Share on other sites

Try to transpose the matrix coming from file (row major vs. column major convention issue)

Try to reverse multiplication order of both matrices (unlikely)

Try combining both approaches.

 

Trial and error fun :)

Edited by JoeJ

Share this post


Link to post
Share on other sites

That only bring a new strange effect, :( see image

Heres again my code

Engine Code

        public static Matrix CalculateMatrixFromParents(Joint joint, Matrix world)
        {
            if (joint.Parent != null)
            {
                world = CalculateMatrixFromParents(joint.Parent, joint.Parent.Transform.Matrix) * world;
                return world;
            }
            else
                return joint.Transform.Matrix;
        }

                List<Matrix> boneMatrices = new List<Matrix>();

                if (this.bones.Count != 0)
                {
                    boneMatrices.Add(Matrix.Identity);

                    foreach (Joint joint in this.bones)
                    {
                        Matrix m = MatrixHelper.CalculateMatrixFromParents(joint, Matrix.Identity);
                        m.Transpose();
                        boneMatrices.Add(m * this.bindPoseMatrix);
                    }
                }

                return boneMatrices;

Shader Code

float4 ApplyBoneTransform(Vertex input, float4 value)
{
 if(HasBones)
 {
	float4x4 skinTransform = (float4x4)0;
	skinTransform += BoneMatrices[input.BoneIndices.x] * input.Weights.x;
	skinTransform += BoneMatrices[input.BoneIndices.y] * input.Weights.y;
	skinTransform += BoneMatrices[input.BoneIndices.z] * input.Weights.z;
	skinTransform += BoneMatrices[input.BoneIndices.w] * input.Weights.w;
	
	float4 position = mul(value, skinTransform);
	
	return position;
 }
 else
   return value;
}

Pixel vertexShader(Vertex input)
{ 
 Pixel result = (Pixel) 0;
 
 float4 posWorld = mul(ApplyBoneTransform(input, float4(input.Position.xyz, 1.0f)), World);
 result.Position = mul(mul(posWorld, View), Projection);
 result.Normal = normalize(mul(ApplyBoneTransform(input, float4(input.Normal.xyz, 1.0f)), WorldIT));
 result.UV = input.UV;
 result.View = ViewInverse[3] - mul(float4(input.Position.xyz, 1.0f), World);
 result.Tangent = normalize(mul(ApplyBoneTransform(input, float4(input.Tangent.xyz, 1.0f)), WorldIT).xyz);
 result.Binormal = normalize(cross(input.Normal, input.Tangent));

 return result;
}

 

Greets

Benjamin

01.jpg

Share this post


Link to post
Share on other sites

Hmm... thinking of it, there might be still something wrong with the import or wiring the data properly.

What is really strange is that the mesh triangles seem to be teared apart. This indicates a logical error more likely than wrong math.

If only math would be wrong, the mesh would bend strangely, but it would not disconnect.

So i assume you have duplicated vertices due to different normals or UVs. You could look for those vertices (log their numbers or using debugger). If you find two vertices with the same position, they should have the same bone indices and weights. If not, import logic must be still wrong.

(After that: I see you draw white lines and circles, so you already have debug visuals. You could use them to do the whole skinning on CPU to find bugs more easily as suggested earlier.)

 

Share this post


Link to post
Share on other sites

Hi Joe,

you had totally right, after hours of checking my code, i found two fatal bugs i made.

The first one was a logic mistake by load the boneindices of the file and the second was to set the wrong offsets for weights and boneindices to the shader.

So i fix this and now the geometry will draw right and can be deform by the bone matrices, without disconnect the triangle shap :D

But there is still a last little bug. If i send the bone matrices in order of the code i post, i get a little scaling effect of the X Axis, so the mesh will get a little bit larger :(

Inverse order of multiplication the bone matrices has the same result. Transpose the value of multiplication of all matrices, the mesh will pressed like from a heavy hammer. Invert the result of multiplication, i get a scaling effect of the X Axis too, but it will be a little bit smaller.

Has anyone an Idea to fix this last step?

Greets

Benjamin

Share this post


Link to post
Share on other sites

If i import the Collada file back to Maya i get a warning message that the tranform of every single joint is not compatible with fbx, so its baked into trs?

Is not the Problem, that the gpu skinning should be only, if bone matrices animations exist, because my 3 bones has transform matrices who say where they should be in the world, but that is not the position of the vertices, so for a example, a bone with the x postion 3 and tranform with the binding vertices translate these 3 steps right on the x Axis and that is wrong right, because without a animation, every single vertex should be only tranform with a matrix ideity until the get animated?

So the transform matrix should be the difference of the old and new bone transform matrix around the postion of the bone?

Greets

Benjamin

Share this post


Link to post
Share on other sites
20 hours ago, JoeJ said:

(After that: I see you draw white lines and circles, so you already have debug visuals. You could use them to do the whole skinning on CPU to find bugs more easily as suggested earlier.)

Maybe it's time for this now. I'd start without any weights and linking each vertex to the closest bone. That's a bit easier but enough to understand verify any involved math.

EDIT:

The inverse bind matrix transforms the vertex from model space to local bone space

The animated bone matrix transforms from bone space to final world space

 

Edited by JoeJ

Share this post


Link to post
Share on other sites

As requested by PM, i'll try to give an example for better understanding...

I don't know how 'new' you are to this, but to me the key was to treat this all as a geometrical problem. Back then i was not aware this is an application of linear algebra, and i still think of this stuff in a pure geometrical way.

Let's start with an exercise of how to transform a vector from one space to another:

// our source space: (referring to the bone transform in rest position)
vec sx (1,0,0);
vec sy (0,1,0);
vec sz (0,0,1);
vec sp (2,3,4);
// and target space: (referring to the animated space)
vec tx (0,0,1);
vec ty (1,0,0);
vec tz (0,1,0);
vec tp (5,6,7);
// this is a skin vertex in global space (or model space, but usually that's the same if model and skelton are at the origin)
vec skin (3,3,3); 

// first, we move the vertex from world space to the local source space
vec localInS = skin - sp; // position now relative to source origin, now care for orientation:
localInS = vec (
	sx.Dot(localInS), // how far from source origin along the direction of its X axis?
	sy.Dot(localInS), // how far from source origin along the direction of its Y axis?
	sz.Dot(localInS)); // and Z
// vertex is now in source space. Key is here to understand how the dot product works.
// so we transformed the vertex from its modeled position to the bone of the skeleton that should affect in during animation.

// next, think of the target space as a animated variant of the skeleton bone.
// we already know the position relative to that bone, now all we need to do is to transform it back to world space but using animated transform
vec animatedSkin = tp + vec (
	tx * localInS.x,
	ty * localInS.y,
	tz * localInS.z);

// that's it. We are done. If we have multiple bones affecting the vertex, we do the same calculation for each of them and add te a weighted final result, like:
vec result = 
	animatedSkin * 0.25 +
	animatedSkin_2 * 0.25 +
	animatedSkin_3 * 0.5;
// that's obvious, but just notice we lerp only resulting vectors istead the spaces, which is faster

If you understand this (take some time, imagine it geomatrically, visualize it to help your brain a lot...),

then you understood all math there is to know.

We could rewrite this code using matrices, the spaces already hint how as they have the same memory order than a 4x3 matrix, just adding a (useless) final row to get 4x4 it would look like this:

	float sMatrix[16] = {
	1,0,0,0,
	0,1,0,0,
	0,0,1,0,
	5,6,7,1};
	

You can then look your math library code to see it performs identical operations than dot products.

Thinking this further, e.g. transforming all 3 direction vectors of one matrix by another matrix, you already understood how 3D rotations work using matrices. Only requirement is to understand the dot product (!)

(This assumes OpenGL matirx order, DirectX transposed that convention because MS likes to have its own standards to force people into their great... you get me)

So in DirectX it would look like this:

	float sMatrix[16] = {
	1,0,0,5,
	0,1,0,6,
	0,0,1,7,
	0,0,0,1};
	

... which causes things like using simple multiplications instead dot product and vice versa - trial and error fun starts here.

 

I'll rewrite above code using matrices:

matrix mS = {...};
matrix mT = {...};
vec skin (3,3,3); 

vec localInS = mS.Unrotate(skin - sp);
// or:
vec localInS = mS.Inversed().Transform(skin);
// or:
vec localInS = mS.InverseTransform(skin);

vec animatedSkin = mT.position + mT.Rotate(localInS);
// or:
vec animatedSkin = mT.Transform(localInS);

Try to follow this so all variants of the same thing makes sense.

Finally, let's combine both involved transforms to one (that's what i've missed in my very first post.)

	matrix combined = mS.Inversed() * mT; // or mT * mS.Inversed(), depending on the convention your math lib uses
vec animatedSkin = combined.Transform(skin);
	

Usually you can write just 'animatedSkin = combined * skin', but 'Transform' makes more sense eventually :)

 

 

So that's it. I hope this helps to understand the math, but i doubt it will help you much to fix your actual problem - this is expected.

There seems nothing wrong with how you do it, bug an be anywhere. I can only repeat my suggestions to do it all an CPU and visualize all steps... good luck! ;)

 

 

Edit: To be sure here is how a space is defined if you're total noob :)

vec sx (1,0,0); // local orientation x axis

vec sy (0,1,0); // local orientation y axis

vec sz (0,0,1);

vec sp (2,3,4); // position

... same for matrices of course

 

 

Edited by JoeJ

Share this post


Link to post
Share on other sites

Hi Joe,

thank you very much again, to take the time to explian me it, but i see in your example, we assume that we animate the model, but whats with the case, i only load the model without animation matrices, so we dont have a target space, just to see the skinned model, do i need than to transform the vertices too, or only if i had animation matrices?

Greets

Benjamin

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!