Following my last, rather angry, post about trying to get the Bullet constraints to work, I took a step back and breathed deeply for a while, then figured out where I was going wrong.

In short, you have to provide two transforms to each constraint, but in 99% of cases (all of mine in the ragdoll sense) these are just the local equivalents of the same transform, just expressed in terms of each rigid body's local space. So I changed the API to just take a single transform, then internally inside the Physics method that adds the constraint, used the inverse of the body transform to work out the two local transforms.

HingeConstraint *Physics::createHingeConstraint(Body &bodyA, Body &bodyB, const Matrix &transform, float min, float max){ Matrix transformA = transform * inverseMatrix(bodyA.transform()); Matrix transformB = transform * inverseMatrix(bodyB.transform()); HingeConstraint *c = new HingeConstraint(); c->hinge = new btHingeConstraint(*(bodyA.bd), *(bodyB.bd), toBulletTransform(transformA), toBulletTransform(transformB)); c->hinge->setLimit(min, max); rep->addConstraint(bodyA, bodyB, c, true); return c;}ConeConstraint *Physics::createConeConstraint(Body &bodyA, Body &bodyB, const Matrix &transform, float swingA, float swingB, float twist){ Matrix transformA = transform * inverseMatrix(bodyA.transform()); Matrix transformB = transform * inverseMatrix(bodyB.transform()); ConeConstraint *c = new ConeConstraint(); c->cone = new btConeTwistConstraint(*(bodyA.bd), *(bodyB.bd), toBulletTransform(transformA), toBulletTransform(transformB)); c->cone->setLimit(swingA, swingB, twist); rep->addConstraint(bodyA, bodyB, c, true); return c;}Then it was just a case of figuring out what the default alignment of the constraint was, so I could rotate it to the one I wanted. For example, a hinge constraint seems to default to run down the Z axis, so needed rotation around the Y axis by pi / 2 to have it run across the X axis for the knees for example:

Matrix m = rotationZMatrix(M_PI_2) * translationMatrix(transformCoord(Vec3(0, -shapes[Head].shape->height() / 2, 0), shapes[Head].transform)); physics.createConeConstraint(*(bodies[Spine]), *(bodies[Head]), m, M_PI_4, M_PI_4, M_PI_2); m = rotationZMatrix(-M_PI_2) * translationMatrix(transformCoord(Vec3(0, shapes[LeftUpperLeg].shape->height() / 2, 0), shapes[LeftUpperLeg].transform)); physics.createConeConstraint(*(bodies[Spine]), *(bodies[LeftUpperLeg]), m, M_PI_4, M_PI_4, 0); m = rotationYMatrix(M_PI_2) * translationMatrix(transformCoord(Vec3(0, shapes[LeftLowerLeg].shape->height() / 2, 0), shapes[LeftLowerLeg].transform)); physics.createHingeConstraint(*(bodies[LeftUpperLeg]), *(bodies[LeftLowerLeg]), m, 0, M_PI_2); m = rotationZMatrix(-M_PI_2) * translationMatrix(transformCoord(Vec3(0, shapes[RightUpperLeg].shape->height() / 2, 0), shapes[RightUpperLeg].transform)); physics.createConeConstraint(*(bodies[Spine]), *(bodies[RightUpperLeg]), m, M_PI_4, M_PI_4, 0); m = rotationYMatrix(M_PI_2) * translationMatrix(transformCoord(Vec3(0, shapes[RightLowerLeg].shape->height() / 2, 0), shapes[RightLowerLeg].transform)); physics.createHingeConstraint(*(bodies[RightUpperLeg]), *(bodies[RightLowerLeg]), m, 0, M_PI_2); m = rotationZMatrix(-M_PI_4) * translationMatrix(transformCoord(Vec3(0, shapes[LeftUpperArm].shape->height() / 2, 0), shapes[LeftUpperArm].transform)); physics.createConeConstraint(*(bodies[Spine]), *(bodies[LeftUpperArm]), m, M_PI_2, M_PI_2, 0); m = rotationYMatrix(-M_PI_2) * translationMatrix(transformCoord(Vec3(0, shapes[LeftLowerArm].shape->height() / 2, 0), shapes[LeftLowerArm].transform)); physics.createHingeConstraint(*(bodies[LeftUpperArm]), *(bodies[LeftLowerArm]), m, 0, M_PI_2); m = rotationZMatrix(M_PI_4) * translationMatrix(transformCoord(Vec3(0, shapes[RightUpperArm].shape->height() / 2, 0), shapes[RightUpperArm].transform)); physics.createConeConstraint(*(bodies[Spine]), *(bodies[RightUpperArm]), m, M_PI_2, M_PI_2, 0); m = rotationYMatrix(-M_PI_2) * translationMatrix(transformCoord(Vec3(0, shapes[RightLowerArm].shape->height() / 2, 0), shapes[RightLowerArm].transform)); physics.createHingeConstraint(*(bodies[RightUpperArm]), *(bodies[RightLowerArm]), m, 0, M_PI_2);Resulting in a ragdoll that behaved as planned and seemed to work okay.

So was finally then ready to start the bit I had been really worried about - mapping the ragdoll positions back to the mesh.



So far, I'm just mapping the body and the head as an experiment, but after lots of failure and random guesswork, I seemed to stumble upon something that works, although I'll admit here I don't entirely understand why.

First, a refresher on the skeletal animation system. The vertices of the skeleton include either one or two bone indices with weights, set up in the model editor. When the mesh is rendered, a vertex shader takes as input a matrix palette, which is basically an array of matrices, each one representing the world position of each bone position and orientation expressed in model local space.

The 0'th element of this array is set to the identity matrix to be used when a slot is not used, and the rest are calculated inside the Skeleton class, based on the current Animation. The Animation contains a list of KeyFrames which define the position and orientation of each bone and a time, so you input a t value, it works the position between two key frames, interpolates the positions by the right amount and then multiples up each matrix by its parent(s) to get a single matrix that applies to all the vertices with those indices.

VS_OUTPUT main(VS_INPUT input){ VS_OUTPUT output = (VS_OUTPUT)0; float4x3 skinning = 0; skinning += bones[input.indices[0]] * input.weights[0]; skinning += bones[input.indices[1]] * input.weights[1]; output.position = float4(mul(input.position, skinning), 1); output.position = mul(output.position, worldviewproj);}Crucially, this puts the model in its pose in model local space, so we then have to multiply the vertices my the model's world matrix to actually rotate and position it in the world.

Now, when we are mapping the ragdoll back, it is slightly different, because the ragdoll positions encode both the pose and the world positions of the parts of the models. So firstly we have to pass an identity matrix as the model world transform, because all of the information will be encoded in the new matrix palette.

The RagDoll class that has so far taken care of creating the bodies and constraining them together now needs to also have a method to yield a matrix palette, and the logic inside the Pc class can check to see whether we are alive or in the rag doll state, and, if the latter, skip the character controller update, set the identity matrix as the world matrix and pass the RagDoll's matrix palette to the vertex shader.

For example, rough-out:

if(!ragDoll) { data.animation.skeleton.setKeyFrame(data.animation.controller.keyFrame(blend, data.animation.map)); node->setTransform(world); node->setPalette(data.animation.skeleton.palette()); } else { node->setTransform(identityMatrix()); node->setPalette(ragDoll->palette()); }Now, the first step is to want to apply just the spine. You might think we just need to set one matrix in the palette for this, but remember that when the animation system generates the matrix palette, it multiplies up all of the matrices by their parents in a recursive manner, so actually to have the model all follow the spine, we have to set every matrix in the palette to the correct matrix to map to the spine.

I played around at length with different offsets and ways to do this, and failed, but eventually I stumbled upon what seems to work perfectly. When I create the shapes and connect the transforms, I'm doing it in local model space so that the constraint limits are correct regardless of the pose the model is in. Once the constraints are created, I then update all the shape transforms with the model world transform (and later will match them to the model current pose here as well).

So it turns out that if I store the inverse of this initial transform, when I am building the palette, I just have to multiply this inverse by the shape's current transform to get the correct matrix palette transform:

MatrixPalette RagDoll::palette() const{ MatrixPalette p; p.push_back(identityMatrix()); for(int i = 0; i < 32; ++i) { p.push_back(offsets[Spine] * bodies[Spine]->transform()); } return p;}As I admitted before, I'm not entirely clear on how this works, but as an experiment, I added this line:

MatrixPalette RagDoll::palette() const{ MatrixPalette p; p.push_back(identityMatrix()); for(int i = 0; i < 32; ++i) { p.push_back(offsets[Spine] * bodies[Spine]->transform()); } p[9] = offsets[Head] * bodies[Head]->transform(); return p;}Where 8 is the Head bone index and +1 is for the identity matrix offset at 0, and the result was as per the video above. The whole body follows the spine, but the head follows the head. Woo hoo.

So, as far as I can grasp, I just need to set up a mapping from animation bone to skeleton shape transform, based on whether there is a separate part for the bone or not. Animation bones that don't have an equivalent in the rag doll need to be assigned to the matrix of the animation bone's parent, either directly or via a chain of moving upwards until a parent with a ragdoll equivalent is found, if you see what I mean.

Then we should be golden. Just on then to the far simpler task of getting the ragdoll shapes to be a better fit to the mesh shapes. Think I might use compound bodies for this - Bullet bodies made up of several convex shapes. For example, the lower leg needs to include a shape that covers the foot better, or the foot will sink into the floor, and a better shape is needed for the head and so on. But this is easy stuff compared to what has gone before.

[EDIT] Was a breeze in the end, to get everything set up properly. We have the ragdoll now fully mapped back to the animation mesh below:



This is just being done manually at the moment:

MatrixPalette RagDoll::palette(const Skeleton &skeleton) const{ MatrixPalette p; p.push_back(identityMatrix()); for(int i = 0; i < 32; ++i) { p.push_back(offsets[Spine] * bodies[Spine]->transform()); } p[skeleton.index("Head") + 1] = offsets[Head] * bodies[Head]->transform(); p[skeleton.index("Left Knee") + 1] = offsets[LeftUpperLeg] * bodies[LeftUpperLeg]->transform(); p[skeleton.index("Left Ankle") + 1] = offsets[LeftLowerLeg] * bodies[LeftLowerLeg]->transform(); p[skeleton.index("Left Foot") + 1] = offsets[LeftLowerLeg] * bodies[LeftLowerLeg]->transform(); p[skeleton.index("Right Knee") + 1] = offsets[RightUpperLeg] * bodies[RightUpperLeg]->transform(); p[skeleton.index("Right Ankle") + 1] = offsets[RightLowerLeg] * bodies[RightLowerLeg]->transform(); p[skeleton.index("Right Foot") + 1] = offsets[RightLowerLeg] * bodies[RightLowerLeg]->transform(); p[skeleton.index("Left Elbow") + 1] = offsets[LeftUpperArm] * bodies[LeftUpperArm]->transform(); p[skeleton.index("Left Wrist") + 1] = offsets[LeftLowerArm] * bodies[LeftLowerArm]->transform(); p[skeleton.index("Left Hand") + 1] = offsets[LeftLowerArm] * bodies[LeftLowerArm]->transform(); p[skeleton.index("Right Elbow") + 1] = offsets[RightUpperArm] * bodies[RightUpperArm]->transform(); p[skeleton.index("Right Wrist") + 1] = offsets[RightLowerArm] * bodies[RightLowerArm]->transform(); p[skeleton.index("Right Hand") + 1] = offsets[RightLowerArm] * bodies[RightLowerArm]->transform(); return p;}May be able to automate this in a way by just setting the minimum mapping up, then using the parent indices of the skeleton joints to figure out the correct mappings for the rest of the bones, so the same code would work with a wider rnage of skeleton structures. But the tough work is now done .

[EDIT 2] Final step added now before a code cleanup, by setting the initial pose of the ragdoll to match the current animation pose:



At the point where I was previously applying the world transform to the bodies, after creating the constraints in local space to maintain the constraint rotations and limits regardless of the pose and orientation, I just now replace the matrices by querying the skeleton's transformed positions rather than bind positions:

Matrix bound(const Skeleton &skeleton, const std::string &na, const std::string &nb){ Vec3 a = skeleton.transformedPosition(skeleton.index(na), identityMatrix()); Vec3 b = skeleton.transformedPosition(skeleton.index(nb), identityMatrix()); return matrixAlignedToSegment(a, b);}RagDoll::RagDoll(/* ... */)// create bodies and constrains in bind for(uint i = 0; i < bodies.size(); ++i) { Matrix m = bodies->transform(); if(i == Spine) m = bound(skeleton, "Chest", "Root"); if(i == Head) m = bound(skeleton, "Head", "Neck"); if(i == LeftUpperLeg) m = bound(skeleton, "Left Hip", "Left Knee"); if(i == LeftLowerLeg) m = bound(skeleton, "Left Knee", "Left Ankle"); if(i == RightUpperLeg) m = bound(skeleton, "Right Hip", "Right Knee"); if(i == RightLowerLeg) m = bound(skeleton, "Right Knee", "Right Ankle"); if(i == LeftUpperArm) m = bound(skeleton, "Left Shoulder", "Left Elbow"); if(i == LeftLowerArm) m = bound(skeleton, "Left Elbow", "Left Wrist"); if(i == RightUpperArm) m = bound(skeleton, "Right Shoulder", "Right Elbow"); if(i == RightLowerArm) m = bound(skeleton, "Right Elbow", "Right Wrist"); bodies->setTransform(m * transform); }Need to tidy this up and stop using the strings directly, do some kind of one time look up to get the indices, then can use some kind of intermediate structure to avoid having to specify the joint pairs twice for each body, but when I'm writing something new and complex, I always bash it out in whatever form is easiest first, then refactor it back into decent code structure once I have it working. Just how I roll.

So just thought this would be a good point to update on the progress and open up for any discussion if anyone is interested. Thanks, as always, for stopping by. Keeping this journal alive is a big part of my motivation and it means a lot to know people are interested.