I have been trying to get this to work for the past few weeks but to no avail. Apparently I must still be missing something.
Some background information first:
I use 2 executables, the first one uses Assimp to import mesh, skeleton and animation data. I save everything in 3 separate binary files.
The second program is a modified Rastertek Tutorial, one of the initial ones with basic lighting, to which I'm trying to add vertex skinning.
Loading a static mesh works perfectly. I've also successfully modified the vertex input signature to include the indices and blendweights.
My binary format is extremely simple and, dare I say, naive. The Assimp structures usually consist of a mNumSomething which I save with a simple ofstream e.g. ofs.write( (char *) &numvtxindices, 4); followed with a for loop that iterates the items and writes them too.
It's very easy to read this data back by getting the number of items, then reading them in a loop and cast to the original or a similar same sized structure.
I also flattened the skeletal node hierarchy to a simple array of parent ids as in this tutorial:
Creating a current pose should really be as simple as iterating this array and for each node:
uint parentid = mHierarchy;
mGlobalPoses = mLocalPoses * mGlobalPoses[parentid];
then upload mInvBindMatrices * mGlobalPoses to the shader. Or so I thought, because nothing works.
I just get Spaghetti Bolognaise whatever combination I try, invert Assimps offset matrix or not, transpose or not etc.
Now it might sound like I'm just trying random hacks but in reality I think I do know that for DirectXmath's row-major matrices the above is the right combination, with Assimp's mOffset aiMatrices transposed to XMMATRIX, and aiQuats order (w,x,y,z) swizzled to XMVECTOR's (x,y,z,w)
It doesn't help that most examples are in OpenGL too.
I haven't interpolated anything yet. It's a single pose for which i created a temporary class, adequately named testpose (lol), and probably used a lot of bad coding practices just to keep it simple. I made every member variable public for easy access when sending the matrix palette,
din't create many separate methods etc. Moreover I have still kept skeleton & anim data together. I'm aware once it works it'll need to be split.
OK, now that this is out of the way let's post the most relevant code.
Let's start with the polygon layout
polygonLayout[0].SemanticName = "POSITION";
polygonLayout[0].SemanticIndex = 0;
polygonLayout[0].Format = DXGI_FORMAT_R32G32B32_FLOAT;
polygonLayout[0].InputSlot = 0;
polygonLayout[0].AlignedByteOffset = 0;
polygonLayout[0].InputSlotClass = D3D11_INPUT_PER_VERTEX_DATA;
polygonLayout[0].InstanceDataStepRate = 0;
polygonLayout[1].SemanticName = "TEXCOORD";
polygonLayout[1].SemanticIndex = 0;
polygonLayout[1].Format = DXGI_FORMAT_R32G32_FLOAT;
polygonLayout[1].InputSlot = 0;
polygonLayout[1].AlignedByteOffset = D3D11_APPEND_ALIGNED_ELEMENT;
polygonLayout[1].InputSlotClass = D3D11_INPUT_PER_VERTEX_DATA;
polygonLayout[1].InstanceDataStepRate = 0;
polygonLayout[2].SemanticName = "NORMAL";
polygonLayout[2].SemanticIndex = 0;
polygonLayout[2].Format = DXGI_FORMAT_R32G32B32_FLOAT;
polygonLayout[2].InputSlot = 0;
polygonLayout[2].AlignedByteOffset = D3D11_APPEND_ALIGNED_ELEMENT;
polygonLayout[2].InputSlotClass = D3D11_INPUT_PER_VERTEX_DATA;
polygonLayout[2].InstanceDataStepRate = 0;
polygonLayout[3].SemanticName = "BLENDINDICES";
polygonLayout[3].SemanticIndex = 0;
polygonLayout[3].Format = DXGI_FORMAT_R32G32B32A32_UINT;
polygonLayout[3].InputSlot = 0;
polygonLayout[3].AlignedByteOffset = D3D11_APPEND_ALIGNED_ELEMENT;
polygonLayout[3].InputSlotClass = D3D11_INPUT_PER_VERTEX_DATA;
polygonLayout[3].InstanceDataStepRate = 0;
polygonLayout[4].SemanticName = "BLENDWEIGHT";
polygonLayout[4].SemanticIndex = 0;
polygonLayout[4].Format = DXGI_FORMAT_R32G32B32A32_FLOAT;
polygonLayout[4].InputSlot = 0;
polygonLayout[4].AlignedByteOffset = D3D11_APPEND_ALIGNED_ELEMENT;
polygonLayout[4].InputSlotClass = D3D11_INPUT_PER_VERTEX_DATA;
polygonLayout[4].InstanceDataStepRate = 0;
// Get a count of the elements in the layout.
numElements = sizeof(polygonLayout) / sizeof(polygonLayout[0]);
// Create the vertex input layout.
result = device->CreateInputLayout(polygonLayout, numElements, vertexShaderBuffer->GetBufferPointer(), vertexShaderBuffer->GetBufferSize(),
&m_layout);
if(FAILED(result))
{
return false;
}
The vertex shader
////////////////////////////////////////////////////////////////////////////////
// Filename: light.vs
////////////////////////////////////////////////////////////////////////////////
#define MAXJOINTS 60
/////////////
// GLOBALS //
/////////////
cbuffer MatrixBuffer
{
matrix worldMatrix;
matrix viewMatrix;
matrix projectionMatrix;
matrix joints[MAXJOINTS];
};
cbuffer CameraBuffer
{
float3 cameraPosition;
float padding;
};
//////////////
// TYPEDEFS //
//////////////
struct VertexInputType
{
float4 position : POSITION;
float2 tex : TEXCOORD0;
float3 normal : NORMAL;
uint4 boneidx : BLENDINDICES;
float4 bonewgt : BLENDWEIGHT;
};
struct PixelInputType
{
float4 position : SV_POSITION;
float2 tex : TEXCOORD0;
float3 normal : NORMAL;
float3 viewDirection : TEXCOORD1;
};
////////////////////////////////////////////////////////////////////////////////
// Vertex Shader
////////////////////////////////////////////////////////////////////////////////
PixelInputType LightVertexShader(VertexInputType input)
{
PixelInputType output;
float4 worldPosition;
float4 skinnedpos = float4(0.0f,0.0f,0.0f,1.0f);
// Change the position vector to be 4 units for proper matrix calculations.
input.position.w = 1.0f;
//transform according to joint matrix palette
skinnedpos += input.bonewgt.x * mul(input.position, joints[input.boneidx.x]);
skinnedpos += input.bonewgt.y * mul(input.position, joints[input.boneidx.y]);
skinnedpos += input.bonewgt.z * mul(input.position, joints[input.boneidx.z]);
skinnedpos += input.bonewgt.w * mul(input.position, joints[input.boneidx.w]);
skinnedpos.w = 1.0f;
// Calculate the position of the vertex against the world, view, and projection matrices.
output.position = mul(skinnedpos, worldMatrix);
output.position = mul(output.position, viewMatrix);
output.position = mul(output.position, projectionMatrix);
//todo: transform normals too but since it doesn't work they can wait
// Store the texture coordinates for the pixel shader.
output.tex = input.tex;
// Calculate the normal vector against the world matrix only.
output.normal = mul(input.normal, (float3x3)worldMatrix);
// Normalize the normal vector.
output.normal = normalize(output.normal);
// Calculate the position of the vertex in the world.
worldPosition = mul(input.position, worldMatrix);
// Determine the viewing direction based on the position of the camera and the position of the vertex in the world.
output.viewDirection = cameraPosition.xyz - worldPosition.xyz;
// Normalize the viewing direction vector.
output.viewDirection = normalize(output.viewDirection);
return output;
}
The code that sends the cBuffers
bool LightShaderClass::SetShaderParameters(ID3D11DeviceContext* deviceContext,XMMATRIX * globalPoses, unsigned int numJoints, const XMMATRIX& worldMatrix, const XMMATRIX& viewMatrix,
const XMMATRIX& projectionMatrix, ID3D11ShaderResourceView* texture, const XMFLOAT3& lightDirection,
const XMFLOAT4& ambientColor, const XMFLOAT4& diffuseColor, const XMFLOAT3& cameraPosition, const XMFLOAT4& specularColor, float specularPower )
{
HRESULT result;
D3D11_MAPPED_SUBRESOURCE mappedResource;
unsigned int bufferNumber;
MatrixBufferType* dataPtr;
LightBufferType* dataPtr2;
CameraBufferType* dataPtr3;
XMMATRIX worldMatrixCopy,viewMatrixCopy,projectionMatrixCopy;
// Transpose the matrices to prepare them for the shader.
worldMatrixCopy = XMMatrixTranspose( worldMatrix );
viewMatrixCopy = XMMatrixTranspose( viewMatrix );
projectionMatrixCopy = XMMatrixTranspose( projectionMatrix );
// Lock the constant buffer so it can be written to.
result = deviceContext->Map(m_matrixBuffer, 0, D3D11_MAP_WRITE_DISCARD, 0, &mappedResource);
if(FAILED(result))
{
return false;
}
// Get a pointer to the data in the constant buffer.
dataPtr = (MatrixBufferType*)mappedResource.pData;
// Copy the matrices into the constant buffer.
dataPtr->world = worldMatrixCopy;
dataPtr->view = viewMatrixCopy;
dataPtr->projection = projectionMatrixCopy;
for(unsigned char i = 0; i < numJoints; i++)
{
dataPtr->joints[i] = XMMatrixTranspose( globalPoses[i]);
//dataPtr->joints[i] = globalPoses[i];
//dataPtr->joints[i] = XMMatrixIdentity();
}
// Unlock the constant buffer.
deviceContext->Unmap(m_matrixBuffer, 0);
// Set the position of the constant buffer in the vertex shader.
bufferNumber = 0;
// Now set the constant buffer in the vertex shader with the updated values.
deviceContext->VSSetConstantBuffers(bufferNumber, 1, &m_matrixBuffer);
// Lock the camera constant buffer so it can be written to.
result = deviceContext->Map(m_cameraBuffer, 0, D3D11_MAP_WRITE_DISCARD, 0, &mappedResource);
if(FAILED(result))
{
return false;
}
// Get a pointer to the data in the constant buffer.
dataPtr3 = (CameraBufferType*)mappedResource.pData;
// Copy the camera position into the constant buffer.
dataPtr3->cameraPosition = cameraPosition;
dataPtr3->padding = 0.0f;
// Unlock the camera constant buffer.
deviceContext->Unmap(m_cameraBuffer, 0);
// Set the position of the camera constant buffer in the vertex shader.
bufferNumber = 1;
// Now set the camera constant buffer in the vertex shader with the updated values.
deviceContext->VSSetConstantBuffers(bufferNumber, 1, &m_cameraBuffer);
// Set shader texture resource in the pixel shader.
deviceContext->PSSetShaderResources(0, 1, &texture);
// Lock the light constant buffer so it can be written to.
result = deviceContext->Map(m_lightBuffer, 0, D3D11_MAP_WRITE_DISCARD, 0, &mappedResource);
if(FAILED(result))
{
return false;
}
// Get a pointer to the data in the light constant buffer.
dataPtr2 = (LightBufferType*)mappedResource.pData;
// Copy the lighting variables into the light constant buffer.
dataPtr2->ambientColor = ambientColor;
dataPtr2->diffuseColor = diffuseColor;
dataPtr2->lightDirection = lightDirection;
dataPtr2->specularColor = specularColor;
dataPtr2->specularPower = specularPower;
// Unlock the light constant buffer.
deviceContext->Unmap(m_lightBuffer, 0);
// Set the position of the light constant buffer in the pixel shader.
bufferNumber = 0;
// Finally set the light constant buffer in the pixel shader with the updated values.
deviceContext->PSSetConstantBuffers(bufferNumber, 1, &m_lightBuffer);
return true;
}
void LightShaderClass::RenderShader(ID3D11DeviceContext* deviceContext, int indexCount)
{
// Set the vertex input layout.
deviceContext->IASetInputLayout(m_layout);
// Set the vertex and pixel shaders that will be used to render this triangle.
deviceContext->VSSetShader(m_vertexShader, NULL, 0);
deviceContext->PSSetShader(m_pixelShader, NULL, 0);
// Set the sampler state in the pixel shader.
deviceContext->PSSetSamplers(0, 1, &m_sampleState);
// Render the triangle.
deviceContext->DrawIndexed(indexCount, 0, 0);
return;
}
The vertex declaration
struct VertexType
{
XMFLOAT3 position;
XMFLOAT2 texture;
XMFLOAT3 normal;
unsigned int blendindices[4];
float blendweights[4];
};
My "testpose" header file
/********************************************************************************************************************************************************************\
*
* This class is only used for testing a frame of an animation e.g. 1 pose at a given keyframe
*
*********************************************************************************************************************************************************************/
#pragma once
#include <DirectXMath.h>
#include <iostream>
#include <fstream>
using namespace DirectX;
using namespace std;
class testpose
{
public:
struct Channel
{
XMFLOAT3 S;
XMFLOAT4 R;
XMFLOAT3 T;
};
testpose(void);
~testpose(void);
public:
unsigned int mNumChannels;
Channel *mChannels;
public:
bool Init(void);
unsigned int mNumJoints;
unsigned int *mHierarchy;
XMMATRIX *mInvbindpose;
XMMATRIX *mLocalPoses;
XMMATRIX *mGlobalPoses;
};
My testpose's code //testpose.cpp
#include "testpose.h"
testpose::testpose(void)
{
mNumChannels = 0;
mChannels = nullptr;
mNumJoints = 0;
mHierarchy = nullptr;
mInvbindpose = nullptr;
mLocalPoses = nullptr;
mGlobalPoses = nullptr;
}
testpose::~testpose(void)
{
if(mChannels != nullptr)
{
delete[] mChannels;
mChannels = nullptr;
}
if(mHierarchy != nullptr)
{
delete[] mHierarchy;
mHierarchy = nullptr;
}
if(mInvbindpose != nullptr)
{
_aligned_free(mInvbindpose);
mInvbindpose = nullptr;
}
if(mLocalPoses != nullptr)
{
_aligned_free(mLocalPoses);
mLocalPoses = nullptr;
}
if(mGlobalPoses != nullptr)
{
_aligned_free(mGlobalPoses);
mGlobalPoses = nullptr;
}
}
bool testpose::Init(void)
{
ifstream ifs("../data/anim1frame.bin", std::ifstream::in | ios::binary);
if(ifs.bad())
{
return false;
}
//------------------Read in a frame of animation--------------------------------------------
ifs.read( (char*) &mNumChannels,4);
if(mNumChannels == 0)
{
return false;
}
mChannels = new Channel[mNumChannels];
for(unsigned int i = 0; i < mNumChannels; i++)
{
ifs.read( (char*) &mChannels[i].S, 3 * 4);
ifs.read( (char*) &mChannels[i].R, 4 * 4);
ifs.read( (char*) &mChannels[i].T, 3 * 4);
}
ifs.close();
//---------------read in bone data such as hierarchy and inverse bind transform-----------------------------------------------
ifs.open("../data/skeleton.bin", std::ifstream::in | ios::binary);
if(ifs.bad())
{
return false;
}
//determine number of joints
ifs.read( (char*) &mNumJoints,4);
if(mNumJoints == 0)
{
return false;
}
mHierarchy = new unsigned int[mNumJoints];
for(unsigned int i = 0; i < mNumJoints; i++)
{
ifs.read( (char*) &mHierarchy[i],4);
}
mInvbindpose = (XMMATRIX*) _aligned_malloc(sizeof(XMMATRIX) * mNumJoints,16);
for(unsigned int i = 0; i < mNumJoints; i++)
{
ifs.read( (char*) &mInvbindpose[i],16 * 4); //4x4 matrix of 4 byte floats
mInvbindpose[i] = XMMatrixTranspose(mInvbindpose[i]);
}
ifs.close();
//-------------------------------------convert the animation from vectors & quaternions to actual SIMD-friendly matrices--------------
mLocalPoses = (XMMATRIX *) _aligned_malloc( sizeof(XMMATRIX) * mNumChannels, 16);
for(unsigned int i = 0; i < mNumChannels; i++)
{
//aiQuaternions that were saved are stored as <w,x,y,z> but XMVECTORS use the <x,y,z,w> convention
float x,y,z,w;
w = mChannels[i].R.x;
x = mChannels[i].R.y;
y = mChannels[i].R.z;
z = mChannels[i].R.w;
XMVECTOR S = XMLoadFloat3(&mChannels[i].S);
XMVECTOR R = XMVectorSet(x,y,z,w);
XMVECTOR T = XMLoadFloat3(&mChannels[i].T);
XMMATRIX Smat = XMMatrixScalingFromVector(S);
XMMATRIX Rmat = XMMatrixRotationQuaternion(R);
XMMATRIX Tmat = XMMatrixTranslationFromVector(T);
XMMATRIX SRTmat = Smat * Rmat * Tmat;
//XMMATRIX SRTmat =Tmat * Rmat * Smat;
//SRTmat = XMMatrixTranspose(SRTmat);
mLocalPoses[i] = SRTmat;
//XMMatrixDecompose(&Stmp,&Rtmp,&Ttmp,SRTmat); //just tried to debug this to see whether data seemed valid, it seemed like it was.
}
//--------------------------------------create global poses from the hierarchy ---------------------------------------------------------
mGlobalPoses = (XMMATRIX *) _aligned_malloc( sizeof(XMMATRIX) * mNumChannels, 16);
for(unsigned int i = 0; i < mNumChannels; i++)
{
if(mHierarchy[i] == -1)
{
mGlobalPoses[i] = mLocalPoses[i];
}
else
{
unsigned int parentid = mHierarchy[i];
mGlobalPoses[i] = mLocalPoses[i] * mGlobalPoses[parentid];
//mGlobalPoses[i] = mGlobalPoses[parentid] * mLocalPoses[i]; //I've tried permutations of this
}
}
//-------------------------------------------------generate final Matrix Palette for the vertex shader ---------------------------------------------
for(unsigned int i = 0; i < mNumChannels; i++)
{
//mGlobalPoses[i] = mGlobalPoses[i] * mInvbindpose[i]; //I've tried permutations of this too and combinations with the above ones
mGlobalPoses[i] = mInvbindpose[i] * mGlobalPoses[i];
}
return true;
}
What model I am using (it's a test model from Assimp itself)
scene = importer.ReadFile( "../../../../../SDKs/assimp-3.1.1-win-binaries/test/models/X/BCN_Epileptic.x",
aiProcess_CalcTangentSpace |
aiProcess_Triangulate |
aiProcess_JoinIdenticalVertices |
aiProcessPreset_TargetRealtime_Quality |
aiProcess_OptimizeGraph |
aiProcess_OptimizeMeshes |
aiProcess_ConvertToLeftHanded
&~aiProcess_FindInvalidData | //recommended on a tutorial
aiProcess_SortByPType);
// If the import failed, report it
if( scene == nullptr)
{
cout << "import failed" << endl;
return -1;
}
How I saved 1 frame of animations, no interpolation yet
//---------------------------------------------------temp 1 frame of animation--------------------------------------------------
cout << "saving one frame of animation" << endl;
ofstream ostmp("anim1frame.bin", std::ofstream::out | ios::binary);
ostmp.write( (char*) &scene->mAnimations[0]->mNumChannels,4);
for(unsigned int chanidx = 0; chanidx < scene->mAnimations[0]->mNumChannels; chanidx++)
{
aiVector3D S,T;
aiQuaternion R;
unsigned int rotidx = 0;
scene->mAnimations[0]->mChannels[chanidx]->mNumRotationKeys > 1 ? rotidx = 45 : rotidx = 0; //not all channels have an array of rotations
S = scene->mAnimations[0]->mChannels[chanidx]->mScalingKeys[0].mValue; //none of them have more than one
R = scene->mAnimations[0]->mChannels[chanidx]->mRotationKeys[rotidx].mValue; //well 1 or 100 depending on the channel, choose 45 for no apparent reason (somewhere mid-anim)
T = scene->mAnimations[0]->mChannels[chanidx]->mPositionKeys[0].mValue; //same as scalings
ostmp.write( (char*) &S,3*4);
ostmp.write( (char*) &R,4*4);
ostmp.write( (char*) &T,3*4);
}
cout << endl;
ostmp.close();
How my skeleton was saved
//------------------------------ save skeleton data-----------------------------------------------------------------
cout << "saving skeleton" << endl;
ofstream ofskel ("skeleton.bin", std::ofstream::out | ios::binary);
unsigned int numnodestosave = flattenedNodes.size();
ofskel.write( (char*) &numnodestosave,4);
for (unsigned int i = 0; i < numnodestosave; i++)
{
ofskel.write( (char*) &flattenedNodes[i].id, 4); // 4 byte ids
}
for (unsigned int i = 0; i < numnodestosave; i++)
{
ofskel.write( (char*) &flattenedNodes[i].invbindpose, 4 * 4 * 4); // 4 bytes per float, 4x4 matrix
}
ofskel.close();
For completeness, I added a Renderdoc screen cap that shows indices and weights seem OK.