# No body cares about me and my Skinning Shader/ Physics Engine problems!

This topic is 3439 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

I'm having a really hard time figuring this out, I am trying to create a rag doll with PhysX. I ridged it right but I can't figure out how to get the matrices to work with my skinning shader. The problem is that I can only get a global world matrix for each bone. I thought I could inverse the offset matrix for the mesh's bones and then multiply it by the global pose of the actor for that bone to get the difference in rotation and position. It works for the position but the rotation is completely messed up and I have no idea how to fix it. If anyone has done rag dolls using any type of physics engine with a soft skinning shader please help me I'm completely lost and have been trying to fix this for days now. [depressed] What I'm doing now with the matrices:
	void Render( D3DXMATRIX mView, D3DXMATRIX mProj )
{
vector<D3DXMATRIX> matrices;
matrices.resize(bones.size());
D3DXMATRIX tmatrix;
for(UINT i = 0; i < bones.size(); i++ )
{
bones->getGlobalPose().getColumnMajor44( matrices );
D3DXMatrixInverse( &tmatrix, NULL, &mesh->bones.matrix );
D3DXMatrixMultiply( &matrices, &tmatrix, &matrices );
}
mesh->Render( matrices, mView, mProj );
matrices.clear();
}


Texture2D txDiffuse
<
string UIName = "Base Texture";
>;
SamplerState samLinear
{
Filter = MIN_MAG_MIP_LINEAR;
};

TextureCube txEnvironment
<
string UIName = "Environment Map";
>;
SamplerState envSampler
{
Filter = MIN_MAG_MIP_LINEAR;
};

float Environment_map_intensity
<
string UIName = "Intensity";
float UIMin = 0.000000;
float UIMax = 1.000000;
float UIStep = 0.01;
string UIWidget = "slider";
> = 1.000000;

float4 Ambient  <
string UIName = "Ambient";
> = float4( 0.47f, 0.47f, 0.47f, 1.0f );

float4 Diffuse  <
string UIName = "Diffuse";
> = float4( 0.47f, 0.47f, 0.47f, 1.0f );

float4 lightPos : Position <  string UIName = "Light Position"; string Object = "PointLight";> ={0.0f, 0.0f, -10.0f, 0.0f};

float4x4 World				: WORLD;
float4x4 View				: WORLDVIEW;
float4x4 Projection			: PROJECTION;
float4x4 Bones[256]		: WORLDMATRIXARRAY;

struct VS_INPUT
{
float4 pos			: POSITION;
float3 norm			: NORMAL;
float2 tex			: TEXCOORD0;
float4 bw			: TEXCOORD1;
float4 bi			: TEXCOORD2;
};

struct PS_INPUT
{
float4 pos			: SV_POSITION;
float4 pos2			: POSITION;
float2 tex			: TEXCOORD0;
float3 norm			: NORMAL;
};

PS_INPUT VS( VS_INPUT In )
{
PS_INPUT Out = (PS_INPUT)0;
float4x4 skinTransform = 0;
skinTransform += Bones[In.bi.x] * In.bw.x;
skinTransform += Bones[In.bi.y] * In.bw.y;
skinTransform += Bones[In.bi.z] * In.bw.z;
skinTransform += Bones[In.bi.w] * In.bw.w;
Out.pos = mul(In.pos, skinTransform);
Out.norm = mul(In.norm, (float3x3)skinTransform);
Out.pos = mul(mul(Out.pos, View), Projection);
Out.tex = In.tex;
return Out;
}

float4 PS( PS_INPUT input) : SV_Target
{
float4 color = txDiffuse.Sample( samLinear, input.tex );
float a = color.w;
color = (color * (dot(input.norm,normalize(input.pos2-lightPos)))) + ( Environment_map_intensity * txEnvironment.Sample( envSampler, -reflect( normalize( input.pos2 - View[3] ).xyz, input.norm.xyz )));
color.w = a;
return color;
}

technique10 Render
{
pass P0
{
}
}


[Edited by - CodaKiller on January 17, 2009 5:31:45 PM]

##### Share on other sites
I'm still having the problem, I thought I fixed it until I used a more complex rig. Has anyone done this before?!

##### Share on other sites
So no one has used a skinning shader before with any type of physics engine?...

##### Share on other sites
What can I do to explain the problem better?

Bump...

##### Share on other sites
Hi,

Probably not of much help. But this is how I do my bone animation.

I calculate the local matrix by interpolating between two key frames (yes, I know this is mathematically not correct, but if the keyframes aren't too far apart appears ok). Then I multiply the local transform with anything from the parent object.

From the top of my head. I export everything from blender (custom exporter) and first multiply with the inverse of the rest pose (so every bone is moved completely 'down' once).

So first multiply with inverse rest pose then with all the transforms from the world and all the transforms in the hierarchy (ParentWorldTransform) to this one.

Note that the order A*B or B*A depends on the way stuff is exported so might be different in your situation.

The best way to debug is take a very simple situation, calculate by hand and verify in debug mode. Tedious but once the code is right you can probably forget about it :-)

void M3DAnimator::Transform( SVRefPtr<M3DEffect> Effect, M3DMatrix& ParentWorldTransform, UInt32 FrameNumber ){	this->FrameNumber = FrameNumber;	M3DMatrix Local;	// Either a regular object or dropped here	if( Object )	{		Object->Frames.MatrixLerp( Local, this->FrameNumber );		D3DXMatrixMultiply( &WorldTransform->GetD3DXMATRIX(), &Local.GetD3DXMATRIX(), &ParentWorldTransform.GetD3DXMATRIX() );		Channel << Level4 << "Parent matrix" << ParentWorldTransform;		Channel << Level4 << "Local  matrix" << Local;		Channel << Level4 << "WorldTransform" << WorldTransform;		for( std::vector<SVRefPtr<M3DAnimator> >::iterator ChildIter = Children.begin(); ChildIter != Children.end(); ++ChildIter )		{			(*ChildIter)->Transform( Effect, *WorldTransform, FrameNumber );		}		for( std::vector<SVRefPtr<M3DAnimator> >::iterator ChildIter = Hidden.begin(); ChildIter != Hidden.end(); ++ChildIter )		{			(*ChildIter)->Transform( Effect, *WorldTransform, FrameNumber );		}	}}

In (one of) my vertex shader I then calculate how different bones( weighted) are pulling on the vertice

SPS VS_BO( SVS_BO i ){	SPS Out;	Out.Pos = 		mul( i.Vertex, InvMulPose0 ) * i.Weights.x +		mul( i.Vertex, InvMulPose1 ) * i.Weights.y +		mul( i.Vertex, InvMulPose2 ) * i.Weights.z +		mul( i.Vertex, InvMulPose3 ) * i.Weights.w;	float3 Normal =		mul(i.Normal, (float3x3) InvMulPose0 ) * i.Weights.x + 		mul(i.Normal, (float3x3) InvMulPose1 ) * i.Weights.y + 		mul(i.Normal, (float3x3) InvMulPose2 ) * i.Weights.z + 		mul(i.Normal, (float3x3) InvMulPose3 ) * i.Weights.w;            Out.Pos = mul( Out.Pos, WorldView );    float3 Norm = mul( Normal, (float3x3)WorldView );        CalcSpot( Norm, Out.Pos, Out.Color, Out.Spec, NrOfSpotLights, true );        Out.Pos = mul( Out.Pos, Proj );      return Out;}

Regards, Ron AF Greve

[/source]

##### Share on other sites
Quote:
 Original post by Ron AF GreveHi,Probably not of much help. But this is how I do my bone animation.I calculate the local matrix by interpolating between two key frames (yes, I know this is mathematically not correct, but if the keyframes aren't too far apart appears ok). Then I multiply the local transform with anything from the parent object.From the top of my head. I export everything from blender (custom exporter) and first multiply with the inverse of the rest pose (so every bone is moved completely 'down' once).So first multiply with inverse rest pose then with all the transforms from the world and all the transforms in the hierarchy (ParentWorldTransform) to this one.Note that the order A*B or B*A depends on the way stuff is exported so might be different in your situation.The best way to debug is take a very simple situation, calculate by hand and verify in debug mode. Tedious but once the code is right you can probably forget about it :-)*** Source Snippet Removed ***In (one of) my vertex shader I then calculate how different bones( weighted) are pulling on the vertice*** Source Snippet Removed ***Regards, Ron AF Greve[/source]

Are you talking about animation matrices or matrices from the physics engine?

##### Share on other sites
I'm not sure if I'm thinking about this right, I need to find the difference between two matrices to get a local matrix. I've been inverting one of them and multiplying it but the other.

I'm not sure if thats how I would do it?

##### Share on other sites
Hi,

Sorry mine was for regular animation I never used the physics engine. On the other hand is there any difference? I would assume that the physix engine just supply you with the final matrices, however since I never used it I might be wrong.

This is how I do it and if I look at your code most of it is in your code to assuming mesh->bones.matrix is the matrix of the bone in 'rest position' (but why isn't it precalculated since that shouldn't change?).

And where is the world transform or is this part all done in model space (then it would seem right)? The rotation would be off if it is not done in model space!

I think to make skinning work you have to have the bone back at the origin (so no world transform), because I figure that is where your vertices are. Now it always takes myself time and a lot of thinking to get it right. But I think you have to multiply the bone with the inverse of yours model world transformation (I'll take it that the pysics engine returns the world transformation?). So it is back in the neighbourhood of the models vertices (also not transformed into world space of course). But the bones stil do contain any transformation.

1. So take the inverse world transformation of you models and apply that to the bones returned from physix.

Now imagine that this is just the rest position. Obviously no transformation of the vertices should happen in this situation. However applying the bone transformation to them would move them i.e. we have to remove the 'rest position' transformation from the bones.

2. Multiply the matrix from the previous step with the inverse rest position of each bone.

Now what you have got is the remainimg transformation. This you can stage to the graphics card (check: for the rest position I would expect this to be the identity matrix).

After that the weighting and moving the vertices to world space would be done.

3. Apply weighting (i.e. the code in your VS is already there). and world/projection/view transformation.

##### Share on other sites
Quote:
 Original post by Ron AF GreveHi,Sorry mine was for regular animation I never used the physics engine. On the other hand is there any difference? I would assume that the physix engine just supply you with the final matrices, however since I never used it I might be wrong.This is how I do it and if I look at your code most of it is in your code to assuming mesh->bones.matrix is the matrix of the bone in 'rest position' (but why isn't it precalculated since that shouldn't change?). And where is the world transform or is this part all done in model space (then it would seem right)? The rotation would be off if it is not done in model space!I think to make skinning work you have to have the bone back at the origin (so no world transform), because I figure that is where your vertices are. Now it always takes myself time and a lot of thinking to get it right. But I think you have to multiply the bone with the inverse of yours model world transformation (I'll take it that the pysics engine returns the world transformation?). So it is back in the neighbourhood of the models vertices (also not transformed into world space of course). But the bones stil do contain any transformation.1. So take the inverse world transformation of you models and apply that to the bones returned from physix. Now imagine that this is just the rest position. Obviously no transformation of the vertices should happen in this situation. However applying the bone transformation to them would move them i.e. we have to remove the 'rest position' transformation from the bones.2. Multiply the matrix from the previous step with the inverse rest position of each bone.Now what you have got is the remainimg transformation. This you can stage to the graphics card (check: for the rest position I would expect this to be the identity matrix).After that the weighting and moving the vertices to world space would be done.3. Apply weighting (i.e. the code in your VS is already there). and world/projection/view transformation.

Hmmm I get what your saying but even though I have a world matrix in the shader it's not actually being used, so I guess I'll have to do a trick to get the world matrix from the root bone which I believe I have an idea as to how I would do it. Alright I'll try that and tell you the results.

• 16
• 11
• 10
• 9
• 49
• ### Forum Statistics

• Total Topics
631392
• Total Posts
2999743
×