Skeletal Animation System

This topic is 1400 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

Recommended Posts

Hi, I'm coding my skeletal animation system for my engine and I'm not really sure what and how I'm supposed to do what I need to do... I have a vague idea, but I can't really put it into the code in a way that I like. I'm having trouble with both the concept and technical parts of it.

I have this structure:

class KeyFrame
{
public:
unsigned int keyFrame;
Vector3 keyValues;
};

class Bone
{
public:
//Other stuff to deal with hierarchy

vector<KeyFrame*> keyTranslates;
vector<KeyFrame*> keyRotates;
};

class Track
{
public:
wstring trackName;
bool trackLoop;
unsigned int frameStart;
unsigned int frameEnd;
};

class Skeleton
{
public:
vector<Track*> tracks;
vector<Bone*> bones;

//skeleton data here
float boneTranslations[MAX_BONES * 3]; //vec3 trans
float boneRotations[MAX_BONES * 4]; //quat rots
};


I'm already loading the data I want, so I have something like "Bone01: keyFrame 5 > TranslateX 0.5, keyFrame 10 > TranslateX -0.5" etc

Each model has a pointer to a skeleton, who has a collection of bones, and I define tracks:

class Model
{
public:
Skeleton* skeleton;
Track* currentTrack; //current playing track
};

//Game Start

//Skeleton::AddTrack(wstring trackName, uint frameStart, uint frameEnd, bool trackLoop);

//Setting this model's current track as WALK
myModel->SetTrack(L"WALK");


So, first I think I need to change the "current track" to support multiple tracks to compose animations (such as WALK + ATTACK) right?

Then the problem I'm having most is figuring my "Update Skeleton" function:

void Model::Draw(Camera* camera, float& interpolation)
{

}

void Skeleton::Update(uint currentFrame)
{
for(uint i = 0; i < bones.size(); i++)
{
//checking ALL translates for that bone
for(uint j = 0; j < bones[i]->keyTranslates.size(); j++)
{
//what should I do here?
//check if this keyFrame value is higher or equal than currentFrame, then loop again
//to search for the previousKeyFrame (if it has one), then interpolate between the two,
//then interpolate again to find the middle (currentFrame)?
//And where/how does deltaTime fits into this? Maybe having a previousFrame as well?

//Actually, scratch this, this seems bad to do at every render call,
//I'd like to avoid doing this if I can and "remember" the animation motion I'm currently at
//for that model so I can skip this operation.
//...or a clever way to handle this
}

//Now check for rotations as well...
for(uint j = 0; j < bones[i]->keyRotates.size(); j++)
{}
}
}


I'm trying to save these operations by remembering the motion I was, but then I'd need to save in the model, for each bone, the previous keyFrame and the next keyFrame for all of them. And I would only need to figure the next motion when I reach the "nextKeyFrame". But this means basically having a copy of the skeleton's bones in the model... not sure if I should go this way.

I've been thinking since yesterday but I can't figure how should I proceed from here... any help?

Share on other sites

You have an architectural problem: It is wrong in principle if a drawing routine has to update the model. When it's time to render, all animation, physics, collision resolution stuff has been done and a stable situation is reached. At best the skinning may be done then, although I would do so only if the skinning is on-the-fly on the GPU.

So, the solution to animating a skeleton (or any other animatable variable) is to have an animation sub-system. The sub-system manages all of the currently active animation tracks. A track is bound to (e.g.) the orientation part of a bone, another track is bound to the position part of the bone (or, if you use combined tracks, simply to the placement of a bone). When the runloop invokes the update() method of the animation sub-system, the active tracks are iterated, the respective key values are interpolated, and the result is written to the animated target.

In practice this is a bit more complex, because of the need for animation blending and perhaps animation layering. (We have a thread here that discusses 2 possible technical solutions and some background.) In the end you need to compute a weighted average of position and orientation for each bone, where choosing the weights decides on blending or layering.

After all animation tracks are processed, also each bone has its current local transform set. The next step is then probably to compute their global transforms w.r.t. the model's space.

You still may want to implement an explicit Skelton::update(…). Notice that this means to have a location where the animation tracks for a particular skeleton are concentrated, but in the end it is important to obey the overall sequence of updates when processing the runloop. You may read this book excerpt over there at gamasutra about the need of a defined sequence of updates. Furthermore, with an animation sub-system that deals with animations in general you no longer have the special case of skeletons but use the system for animating other variables, too.

Share on other sites

You have an architectural problem: It is wrong in principle if a drawing routine has to update the model.

Oh but that's been done already, it's that I want to use interpolation for the movements of the bones as well, that's why I put the update inside the Draw() function.

The currentFrame will be changed during the update, and I want to get the previousFrame > currentFrame interpolation during the draw. But I still haven't done the regular updates so I left the deltaTime/interpolation out and forgot to mention it, I'm sorry about that.

After all animation tracks are processed, also each bone has its current local transform set. The next step is then probably to compute their global transforms w.r.t. the model's space.

If I understood correctly, then I'm supposed to bind all affected bones to a track, then I can iterate only over the affected ones of the current tracks. That seems better than searching the whole skeleton again, I'll do that, thanks!

But still, there's the issue of figuring out where the model is at a time. I was thinking in "saving" the current time on each bone, for example, so for example I have an animation called "WALK" that are frames 5 to 10, and I'm currently at 6, I'll have references to both the keyFrame 5 values and keyFrame 10 values so I don't have to look up it again all the time.

But this idea crumbled when I realized that I can't "save" these references on the bones because of the situation when multiple models share the same skeleton and have different times. Then I went to the idea of having a reference of all bones and keyframes per model, but that didn't seemed right since I'm having to allocate a new bone/keyFrame structure per Track set on the model...

I'm trying to figure if there's some really clever way to deal with all of these either with some algorithm to handle the figuring out of the current keyframe and interpolation values, or using a new/changing structure to better handle all this.

And thanks for the link on blending, I'll definitely want to implement something like that and that'll be a good read.

Share on other sites

You may not be able to view this article yet, but it might be some good info for you. It's currently in moderator review and should be "released" within a week or two.

In any case, one concept of an animation controller class is briefly as follows:

- The hierarchy maintains data for the character's rest (or pose) position.

- An animation controller maintains data for the character in an animated position.

- The animation controller "data" is comprised of sets of animations, each "animation" being a set of timed key frames associated with a name.

- The animation name does and must represent a bone name in the hierarchy. However, ...

- An animation controller knows nothing about a "hierarchy" and needs only information to ensure the names in the animation data match a list of names. That list of names happens to be a list of bone names from the hierarchy, but that isn't material.

- The animation data set has a period (length of time for the animation set) associated with it. E.g., a 3-second walk sequence. The animation set data and its period is a fixed set of data, and remains so for the life of the animation set.

- The animation controller maintains a set of "tracks," each track having a set of animations associated with it.

- The track maintains the "local" or "current" time of the animation set. That local time (normally) runs from 0 to the period of the associated animation set.

An animation controller may be setup to update either a single animation set, or blend two (or more) animation sets. Blending is the process of (for instance) transitioning from "walk" to "run."

The animation controller has (e.g.) an AdvanceTime( deltaTime ) function. That function checks all the tracks, checking whether they are enabled. For those tracks which are enabled, the track's current time is updated by deltaTime ( currentTime += deltaTime). After some checks on the currentTime of the track versus the period of the associated animation set, a routine is called which interpolates the animation set data appropriate for the current time, and stores the results in a buffer belonging to the track. The buffer is sized to hold interpolated key frame data for every name (in the hierarchy) for a single moment in time, the current time for the track.

When all tracks have been updated, if just a single track is enabled, the interpolated key frame data is used to calculate (for instance) the animation transform (matrix) for a bone and store the matrix in the appropriate place.

If a blend is in progress, then the data in the two appropriate track buffers is interpolated (or in some other way combined), and the animation transform calculated from that result.

EDIT: Note, fixed data is maintained by the hierarchy and the animation sets. Data which changes (times, matrices) is maintained separately.

Edited by Buckeye

Share on other sites

But still, there's the issue of figuring out where the model is at a time. I was thinking in "saving" the current time on each bone, for example, so for example I have an animation called "WALK" that are frames 5 to 10, and I'm currently at 6, I'll have references to both the keyFrame 5 values and keyFrame 10 values so I don't have to look up it again all the time.
But this idea crumbled when I realized that I can't "save" these references on the bones because of the situation when multiple models share the same skeleton and have different times. Then I went to the idea of having a reference of all bones and keyframes per model, but that didn't seemed right since I'm having to allocate a new bone/keyFrame structure per Track set on the model...

I'll describe the procedure I meant with more details.

An animation like WALK is a resource of the game. It provides a couple of tracks. Each track has a type which defines what kind of variable can be driven. That type may be a vector3 which can hence drive a variable of type vector3. The type may be a quaternion, it may also be a bigger compound like an entire placement. Perhaps the type is tagged with a semantic like position, orientation, normal, and so on; this would give a better control of what a track is good for. The animation itself has a duration. All of its tracks can deliver values for the entire duration. The animation is fix over time, it has no update (remember that it is a resource). Having a RUN besides WALK means that there is a second animation resource.

The set of tracks in an animation are not necessarily dedicated to drive the placement of bones of a skeleton. Instead their type (and perhaps semantic) defines what they are able to drive on a more abstract level. Furthermore there is not necessarily one track per variable which is exposed by a skeleton. It it totally fine if an animation drives only a subset of available variables, simply because there are tracks for only a few of them. This is useful e.g. to have an influence on legs and the hips only.

At runtime there is an PlayAnimation object. Whether this exists in a blend tree or whatever is a question outside of the scope of this post. The PlayAnimation holds a reference to a single animation resource, and the instance specific parameters that are needed to request the tracks of the referred animation for interpolated values. It has further a binding structure that tells which track is bound to which variable (of a bone, in your case). That means that all things like animation start time, playback speed adaption (is needed for blending cyclic animations like WALK and RUN to FASTWALK), and so on are managed therein.

Updating the animation then means to call the PlayAnimation objects. Those compute the normalized time from the current time, animation start, speed adaption, and  animation duration (and perhaps cycle count). It then iterates the bindings, and requests each of the bound tracks to compute its value w.r.t. the normalized time, and transfers the result to the also bound target variable. The transfer is not just a set. The thread linked to in a post above has details about this.

So a particular animation exists once as a resource. At runtime there need to be one instance of PlayAnimation per currently played animation. If using a blend tree, then all needed PlayAnimation objects exist ever, namely as leaf nodes of the tree, but may be set inactive.

A skeleton as a prototype exists once as a resource if you want so. At runtime you need a storage for the placements of the bones. This is written to for preparation and when the animation updates. Of course, also the conversion from local to skeleton (or model) global transforms happens here.

Share on other sites

Also helpful for you could be the tutorial I added in the graphics programming section a while ago:

I've done GPU skinning already, but it was kinda hard to find resources (I think I only found 2 or 3 explaining the whole process) so it's nice to see more resources about it coming up.

And your tutorial made me notice something I completely forgot about until now, I don't have any Bone "presence" in the scene. My bones don't have anything, actually, so they can't be used as elements to be parents or childs. The transformations I have stored are local only because I thought "since it's index/weight are already defined, I only need that".

This changes a bunch of things for me since I was trying to have a single Skeleton instance for multiple models and that was restraining some options...

I was trying to have this Skeleton class, with bones only having their keyFrames values, and I would calculate it's position/rotations according to each model's animation tracks, then upload the data, since the GPU is doing the skinning and all the indexes/weights are already defined, that was the only thing I needed to do (change the bones, upload).

Really, thanks! I'd be really frustrated if I only realized this AFTER I finished this hahaha

Share on other sites

My bones don't have anything, actually, so they can't be used as elements to be parents or childs.

They should inherit from the same base class that gives the rest of your scene “scene graph” capabilities. A gun should be attachable to any joint as easily as it should to any game object or part of a game object.

Additionally, a keyframe is a time value and a single floating-point value pair. Not a vector. A track is a collection of keyframes. Since a keyframe can only modify a single value, you need 3 tracks to handle rotation, 3 more for translation, etc.

L. Spiro

Share on other sites

You have a Pose skeleton, for example a human skeleton. This should be outside the entity so that many humans in the scene can use the same Pose skeleton, this prevents memory duplication.

A pose skeleton has an array of Pose bones. Each Pose bone has a matrix plus a reference to a parent Pose bone. Some systems also employ sibling bones, but you dont have to.

An animation track has an array of bone tracks. Each Bone track has an array of Key Frames. For each Bone track Find two key frames and interpolate between them, then combine that result to the corresponding Pose bone in the Pose skeleton and save the animated bone to a new location. Now combine each animated bone with the parent animated bone. You combine from top to bottom and you don't need another copy and can reuse the same memory location. The resulting animated skeleton is copied to the GPU and used for skinning.

The animated bone dont need a reference to the parent and can thus be represented only as a single matrix. Such that you can have a std::vector<matrix>. This whole array can then after parent combination be directly copied to GPU without having the reference bytes tag along.

Many edits now

You can have some animation tracks which do not have all bone tracks in them to animate all bones. For example you can have a track which only animate legs. What I do is that I copy the Pose Skeleton (only matrix part, not reference) to a std::vector<matrix> AnimationSkeleton. Then apply every interpolated matrix to AnimationSkeleton. Then you combine the whole array, using the references in Pose Skeleton. The matrix at index 0 is the root bone and so fourth. A bone has always a higher index then all of its parent bones. That way you can traverse the array in one sweep.

Edited by Tispe

Share on other sites
Additionally, a keyframe is a time value and a single floating-point value pair. Not a vector. A track is a collection of keyframes. Since a keyframe can only modify a single value, you need 3 tracks to handle rotation, 3 more for translation, etc.

IMHO: Technically there is no reason why a keyframe cannot be a <time,placement> pair, for example. The constraints that are added by doing so are

a) position and orientation (or whatever is represented by the keyframe's value) are set as a whole; if one wants to animate them separately they need their own keyframes and hence tracks;

b) the keyframes count for the entire value, even if parts of the value (for example the position) would also exactly result from interpolation between the surrounding keyframes, so that for those parts the keyframe would be not needed.

Edited by haegarr

Share on other sites

A keyframe can consist of only a rotation part if all keyframes are evenly seperated. Right?

Share on other sites

Technically there is no reason why a keyframe cannot be a pair, for example.

There is no technical reason, but your mentioned primary 2 constraints proved to me in the distant past too constraining for any practical use.
I originally thought it would be easier to handle interpolation of vectors as their own type but I found out I was just making more pain for myself than necessary. A single track system with only one pairing type is not only easier to implement, it works how the underlying modeling software works which causes fewer animation bugs/restrictions/future work-arounds. It leaves you with only one type of data in your animation file.

Advancing 3 tracks instead of 1 is going to give you slight overhead but in practice you won’t be animating all 3 vector components at once that often, so updating and interpolating 2 tracks (XY for example) separately roughly evens out compared to updating 1 track but interpolating all 3 components (XYZ), with 1 (Z) being unnecessarily updated.

A keyframe can consist of only a rotation part if all keyframes are evenly seperated. Right?

How evenly they are separated should not be a factor.
A keyframe can technically have whatever you want. It could be a 3-component vector indicating radians along each axis, or a 4-component quaternion, or the upper 3×3 part of a transform matrix.

It’s just that the more you deviate from how your authoring software works the more likely you are to have improper animation playback, in addition to multiple implementations for track interpolation, multiple types of data in your file format, etc.

It is generally not worth using anything but a single float and (up to) 3 separate tracks, as counter-intuitive as that sounds.

L. Spiro Edited by L. Spiro

Share on other sites

They should inherit from the same base class that gives the rest of your scene “scene graph” capabilities. A gun should be attachable to any joint as easily as it should to any game object or part of a game object.

I'm trying to avoid inheritance and virtuals so I did this on composition, but I don't know if this will be a problem in the future, so far, it's been manageable:

class SceneObject
{
public:
Orientation currentOrientation;
Orientation previousOrientation

SceneObject* parentObject; //SceneGraph

void UpdateMatrix(float& interpolation);
Matrix4x4& GetMatrix();
//etc, various functions required to "work" the calculations required for the scene positioning
};

class Scene
{
public:
private:
vector<Model*> _models;
vector<Image*> _images; //separated by type, I found it easier to sort and apply commands before drawing

vector<SceneObject*> _objects; //but before drawing, I update all objects' matrices
};

class Model
{
public:
//etc
void Draw(Camera* camera);
private:
SceneObject _sceneObject;
};

class Image
{
public:
private:
SceneObject _sceneObject;
};


And I'm kinda obsessed about performance, so I'm really trying to make the engine do only the minimum required work, so I'm trying to figure if I really really need to have the skeleton in the "scene".

All objects in the scene have their matrices calculated, interpolated, checked for hierarchy (I have some trouble with this too, will describe below) etc

If a skeleton is used only to move a model's vertex data, I don't think I need to keep track of each bone's position in the scene. So I'm trying to figure if it'll be worth it or not.

I'd implement a Bone object that can be held in the Scene, but I just wouldn't put ALL skeleton bones and update all of them for all skeletons. I'd leave it up to the game to know if the "HandBone" is required as an object, then he gets a new instance from the skeleton, attach it to a model, link to the Bone, and it joins the Scene and is an object able to hold children/parents.

IMHO: Technically there is no reason why a keyframe cannot be a pair, for example. The constraints that are added by doing so are
,placement>

I'm thinking in something a bit different, to avoid calculating the track's next keyframe and interpolate, since I'm using "Frames" as time unit for the animations, I'm thinking in doing something like a "Motion" class.

When the skeleton/animation is loaded, I'd check all bones, all keyframes, and I'd create a Motion class with a value begin/end, like:

class Motion
{
public:
int frameStart, frameEnd;
Vector3 position;
Vector3 rotation;
int boneIndex;

Motion* nextMotion;
}


Then when my animation has a track playing, I wouldn't deal with figuring keyframes and such, I'd pick the current Motion, sum the values to the Bone's position if the "frame" time is between the motion. I'd probably be able to do something like this:

//for each track, loop through the motions
for(unsigned int i = 0; i < motion.size(); i++)
{
Bone* bone = skeleton->GetBone(motion->boneIndex); //Getting the bone ref
bone->position += motion->position;
bone->rotation += motion->rotation;
}


(By the way, this only if each model HAS it's own skeleton, I know it conflicts with my idea up above of keeping only one instance, but it's just that I'm trying a bunch of things)

This also provides blending since I can create new motions based on two keyframes, it's just that I don't have to calculate each one during play, except for blends.

There's so many possibilities and I don't know which one to follow...

(I know about premature optimization, but that's how I'd like to do this... I'm not in a hurry to code a game, and I'm using this engine to learn a lot about programming, so I'm doing step-by-step, slowly, understanding the concepts and implementing whatever I figure it's best)

And thanks for the replies, found so many things I've overlooked and so much more to think about and implement!

Share on other sites

If I understand correctly, you design seems to tightly couple a number of distinct concepts that may lead to you duplicating a lot of data due to the denormalised schema. I think it's probably quite important to separate out the concepts of a Skeleton, an Animation and the current PlayState of an animation to factor out shared/reusable data.

The idea would be that Skeletons and Animations don't actually change once they're loaded, i.e. a playing animation (or an animating skeleton) won't 'know' that they're being played/animated and they don't keep track of that themselves. This would be the responsibility of the PlayState to keep track of that kind of thing.

This would mean that a single Skeleton can be shared amongst many animations and entities, animations can be shared amongst many entities and each entity can just keep its own PlayState that records the current position in the timeline. So, for example, you could have 50 humans using the "Human" Skeleton, each one of those guys can be playing their own individual animations or some may be playing the same animation but in any case *all* of those entities can be at any point in time in their playback.

You could extend that design to achieve various effects like blending. For example a PlayState could be configured with a mask to block out the waist-to-feet portion of an animation which frees up the legs for another PlayState to animate. If masks are weighted (rather than boolean) then they could be used to feather the two animations together smoothly around the waist.

In case it isn't clear:

Skeleton - Models the hierarchy of bones, as well as their initial physical properties (bone length, initial transform, name, anything else). It can probably achieve that in an efficient 'flat' format without need for a recursive Bone class, merely some std::vectors for the various properties e.g. std::vector<double> boneLengths;

Animation - A series of keyframes per bone. You might make this as a container of Tracks (where a Track is a series of keyframes), one Track per bone.

PlayState - Pairs a Skeleton with an Animation and maintains the current point-in-time of the playing animation. Obviously it could also cache the current & next keyframes for efficient interpolation.

The final calculated state of an animating skeleton could be spat out as a new Skeleton object with its bones in the correct pose for that frame. Or, if you don't need the hierarchy, then as a new concept: The Pose. Which just models the final transformed state of each bone. You might even refactor Skeleton to be a pairing of a Hierarchy with the skeleton's initial bind Pose.

Share on other sites

I'm thinking in something a bit different, to avoid calculating the track's next keyframe and interpolate, since I'm using "Frames" as time unit for the animations, I'm thinking in doing something like a "Motion" class.
When the skeleton/animation is loaded, I'd check all bones, all keyframes, and I'd create a Motion class with a value begin/end, like:
class Motion
{
public:
int frameStart, frameEnd;
Vector3 position;
Vector3 rotation;
int boneIndex;

Motion* nextMotion;
}
Then when my animation has a track playing, I wouldn't deal with figuring keyframes and such, I'd pick the current Motion, sum the values to the Bone's position if the "frame" time is between the motion. I'd probably be able to do something like this:
//for each track, loop through the motions
for(unsigned int i = 0; i < motion.size(); i++)
{
Bone* bone = skeleton->GetBone(motion->boneIndex); //Getting the bone ref
bone->position += motion->position;
bone->rotation += motion->rotation;
}
(By the way, this only if each model HAS it's own skeleton, I know it conflicts with my idea up above of keeping only one instance, but it's just that I'm trying a bunch of things)

Using frames and avoiding interpolation will most probably show you choppy animations, since the timing of your game loop is required to be absolutely stable. Moreover, it probably causes many more keyframes than are needed. And it hinders you from blending cyclic animations in general.

For the latter argument it must be clear that the playback speed is not totally defined by the animation itself. Let's say we have a WALK animation with 10 frames per cycle, as well as a RUN animation with 5 frames per cycle. Obviously, the RUN animation completes its cycle twice as fast as the WALK animation, and hence the skeleton is moved with double speed. We now want the skeleton to change movement from WALK to RUN in a smooth transition of 2 seconds, or else we want it to run constantly at a speed in-between. Either way, let's say the current speed is 6 frames per second. So you need to accelerate the playback speed of WALK by 10/6 and to decelerate those of RUN by 5/6. If you don't do so, then the one animation shows the phase where the right foot is on ground, while the other animation shows it in air, and this also weaves over time. Together the animation is terrible to look at. Notice that the playback speeds are adapted in a way that easily causes in-between hits w.r.t. keyframes.

Dealing with real keyframes at "arbitrary" time moments is not so bad, since the current pair of keyframes can be memorized in the playback speed object. So yo just need to look forward, e.g. whether the current time delta has exceeded the later of the both keyframes.

If I understand correctly, you design seems to tightly couple a number of distinct concepts that may lead to you duplicating a lot of data due to the denormalised schema. I think it's probably quite important to separate out the concepts of a Skeleton, an Animation and the current PlayState of an animation to factor out shared/reusable data.

...

Seconding.

Edited by haegarr

Share on other sites

If I understand correctly, you design seems to tightly couple a number of distinct concepts that may lead to you duplicating a lot of data due to the denormalised schema. I think it's probably quite important to separate out the concepts of a Skeleton, an Animation and the current PlayState of an animation to factor out shared/reusable data.

Oh that's what I was thinking in, but I wasn't really getting the idea behind it and I was mixing what Animation and PlayState is, thanks for clarifying that for me.

Using frames and avoiding interpolation will most probably show you choppy animations, since the timing of your game loop is required to be absolutely stable. Moreover, it probably causes many more keyframes than are needed. And it hinders you from blending cyclic animations in general.

In the example I forgot about interpolation, but I didn't meant I wouldn't do it, I'm using a double as "frameTimer" and I'd multiply the value by that value (which should be between 0~1). Each update increases the "frameTimer" by a value depending on my "updates per second" timer.

Dealing with real keyframes at "arbitrary" time moments is not so bad, since the current pair of keyframes can be memorized in the playback speed. So yo just need to look forward, e.g. whether the current time delta has exceeded the later of the both keyframes.

Now it seems much more clearer how I can implement this, I wasn't really sure about the "PlayState" class and was trying to fit the Animation (dealing with it as if it wasn't a resource) with it and the code was getting just messy and confusing.

I'm only unsure if I should make the skeleton / bones as a scene object as well, or if I should leave them as only a reference to all "human models" in the scene for example.

I think it might not be so bad calculating and updating all bones' matrices even if I'm not really using them in the scene, and I think I might be able to use them for physics/collisions easier if they're all in the scene (I haven't even started physics so I'm not sure, I'm only trying to plan ahead how I'd do it).

Then implementing a "special case" when a game requires a bone to hold a children in the scene is looking bad since if I'm going to use them for anything else, it wouldn't be the "special" case anymore.

Again, many thanks for all the replies, I got a much clear vision of how I should deal with animations now.

Share on other sites

I structure this like:

Skeleton - defines the joint positions in bind pose

KeyFrame - defines the transforms for a single pose from the animation

Animation - stores a list of keyframes

Animation has a interpolatedKeyFrame(float t) method that returns the keyframe for that point

Skeleton has a palette(const KeyFrame &k) method that returns a matrix palette based on the bind pose and the input keyframe transforms

My equivalent to the PlayState that dmatter mentions in this system is just the float t input to Animation::interpolatedKeyFrame. This float is the only thing that changes once everything is loaded and multiple objects can share the same Skeleton and Animation without issues.

Blending based on fixed update, variable frame rate is handled by just storing a previous t and interpolating.

Animation a;
Skeleton s;

float t;
float previousT;

void render(float blend)
{
KeyFrame k = a.interpolatedKeyFrame(interpolate(previousT, t, blend));

renderMesh();
}

Edited by Aardvajk

Share on other sites

Interesting stuff, Spiro, glad you posted that. I was about to start work on some separate stuff for interpolating single values but you've made me think that could be incorporated into the animation system quite well if, as you say, it is decoupled from the skeleton.

Thanks.

Share on other sites

(as I said before, these really should be just single float values)

There are of course some tradeoffs being made by that choice. It'll work fine for basic types like vectors and euler angles but not as well for types with less separable components such as quaternions and matrices (and they also have to stay normalised). You could perform an operation like NLerp on them, but not for example a SLerp. I guess you could have specialised tracks if complex interpolation was important (ScalarTrack, QuaternionTrack, etc).

Share on other sites

It'll work fine for basic types like vectors and euler angles but not as well for types with less separable components such as quaternions and matrices (and they also have to stay normalised).

It’s not really an issue because in fact by working with tracks in this way you are doing what the authoring software (Maya for example) is doing under its hood, meaning you are actually going to stay closer to the real animation.

In Maya, for example, you have an X, Y, and Z rotation. 3 tracks.
These 3 components can be any values each.

As mentioned near the end of the workflow, if you need a rotation matrix or a quaternion, you generate them from these 3 values (since these are the native 3 values of the authoring software it is guaranteed you have some method to convert these values into matrices or quaternions or whatever else you want to use) as a post-processing step after the animations are finished.

It is more efficient for at least 2 reasons (using a matrix as an example but it applies mostly to quaternions as well):
• Because the rotation is broken into 3 tracks you don’t update components of the rotation that are not actually being animated.  A track that works on full matrices has to update rotation, scale, and translation all together if just one component anywhere changes.
• A track working on full-sized matrices must interpolate from matrix to matrix (same for quaternions).  In the case of matrices this means decomposing, interpolating scale (vector), rotation (quaternion), and translation (vector) separately, and recomposing the matrix.  In the case of quaternions interpolation is a fairly complex function call with a branch for small angles.  In both cases it is easier to simply rebuild the primitive (matrix or quaternion) after doing basic linear scalar interpolation.
Here is a practical example to illustrate how much more efficient this flow is.
As I mentioned our joints have XYZ translations, XYZ scales, and XYZ rotations, each stored separately (they could be 3 vectors or 9 floats; if using vectors, a Track will simply attach directly to the .x, .y, or .z of any given vector, modifying only one component at a time) and the rotational part of the matrix is made by rotating around the axis in XYZ order (first the X axis, then the Y, then the Z).

To construct a rotation matrix around the XYZ axes is as follows:

		CMatrix4x4Base<_tType, _tVector3Type, _tVector4Type> & LSE_FCALL MatrixRotationXZY( _tType _tX, _tType _tY, _tType _tZ ) {
LSREAL fS1, fC1;
CMathLib::SinCos( static_cast<LSREAL>(_tX), fS1, fC1 );

LSREAL fS2, fC2;
CMathLib::SinCos( static_cast<LSREAL>(_tY), fS2, fC2 );

LSREAL fS3, fC3;
CMathLib::SinCos( static_cast<LSREAL>(_tZ), fS3, fC3 );

_11 = fC1 * fC3;
_12 = fS1 * fS3 + fC1 * fC3 * fS2;
_13 = fC3 * fS1 * fS2 - fC1 * fS3;
_14 = _tType( 0.0 );

_21 = -fS2;
_22 = fC1 * fC2;
_23 = fC2 * fS1;
_24 = _tType( 0.0 );

_31 = fC2 * fS3;
_32 = fC1 * fS2 * fS3 - fC3 * fS1;
_33 = fC1 * fC3 + fS1 * fS2 * fS3;
_34 = _tType( 0.0 );

// Zero the position.
_41 = _tType( 0.0 );
_42 = _tType( 0.0 );
_43 = _tType( 0.0 );
_44 = _tType( 1.0 );
return (*this);
}
In Maya you have animated only the Rotation.X from 0 to 360.

So only a single track is updating for the entire joint.
So the entire workload is as follows:
• Update 1 track, interpolating a single float value.
• In post-processing after animation is finished updating, reconstruct matrix as shown below.

• float fRotX = DEG_TO_RAD( ROT.x );	// Can be optimized so that the tracks do this conversion only when they update values so that non-modified values are already in radians.
float fRotY = DEG_TO_RAD( ROT.y );
float fRotZ = DEG_TO_RAD( ROT.z );

float fS1, fC1;
CMathLib::SinCos( fRotX, fS1, fC1 );	// Calculates sin and cos in 1 instruction.
float fS2, fC2;
CMathLib::SinCos( fRotY, fS2, fC2 );
float fS3, fC3;
CMathLib::SinCos( fRotY, fS3, fC3 );

_11 = (fC1 * fC3) * SCALE.x;
_12 = (fS1 * fS3 + fC1 * fC3 * fS2) * SCALE.x;
_13 = (fC3 * fS1 * fS2 - fC1 * fS3) * SCALE.x;
_14 = 0.0f;

_21 = -fS2 * SCALE.y;
_22 = (fC1 * fC2) * SCALE.y;
_23 = (fC2 * fS1) * SCALE.y;
_24 = 0.0f;

_31 = (fC2 * fS3) * SCALE.z;
_32 = (fC1 * fS2 * fS3 - fC3 * fS1) * SCALE.z;
_33 = (fC1 * fC3 + fS1 * fS2 * fS3) * SCALE.z;
_34 = 0.0f;

_41 = TRANS.x;
_42 = TRANS.y;
_43 = TRANS.z;
_44 = 1.0f;
This is clearly faster than interpolating full matrices.

In the end, it’s simply more efficient and it has no drawbacks related to more complex types such as matrices or quaternions.  These are rebuilt as the last step of the flow.

L. Spiro Edited by L. Spiro

Share on other sites

KeyFrame: A single {Time, Value} pair. Can act on an AnimatedSkeleton or anything else; just give it a reference to the value it modifies and another to a dirty flag.

I was going on using KeyFrame with 3 values because I'm using 3DS Max to get my models data, and from what I could get from the files is that if I translate anything on the X-axis, it would create a translate channel (track) for X, Y and Z (with zeros on the unused values).

But animating anything on screen is really great, I hadn't thought about it, thanks!

One question though, I'm supposed to have then an Animation class with a vector of tracks, and these tracks point to a value that they change, and each track has a vector of KeyFrames, the values at which time changes... I'm loading from a FBX file and pairing it with the base Skeleton I get from the file, so the reference values for each track are those bones (like Track #1 is translate X for Bone 1), but then I copy the skeleton to a "skeletonInstance" for the animated model, and the references would still be pointing to the original "pose skeleton".

How can I get to correctly assign the references without making the KeyFrame / Track classes specifically for the Bone/Skeleton structures?

For now I'm having each KeyFrame (changing to track in your structure) assigned to a bone index (int value, not ref), so I know which is which.

And this also means I have to change my loader... ugh!

Share on other sites

If you have noticed, the system I have designed is exactly how FBX works, so you can learn a lot about how animations, tracks, key frames, etc. all work inside a game by looking at how the Autodesk® FBX® SDK works.

To attach a track to the correct parameter of the correct object, it needs the identifier of that object (it’s name tree, which is all the names of its parents in order down to the joint’s name) and a special identifier to indicate which parameter of that object the track is to modify (ComponentId).  You will notice that a switch case with that identifier is how the Autodesk® FBX® SDK knows to which component to attach a track.
A switch case is sadly the only way, but it’s really no big deal.

For now I'm having each KeyFrame (changing to track in your structure) assigned to a bone index (int value, not ref), so I know which is which.

There doesn’t need to be an association between tracks and joints (or any other object that could be animated) at all.
The name tree simply allows to look up the target object and then the switch case allows to find which component the track will modify on that object, and once the track has a reference or pointer to that component you are done. No other associations need to exist.

It looks like this:

Skeleton has a hierarchy of joints. Inside a scene there can be joints of the same name, so the track has a name tree (names of all the objects from the root node down to the target joint (or any other type of object)) to allow it to locate the object/joint it will animate.
Once found, that object/joint can use a virtual interface to take the track and its ComponentId and allow the object to connect the track to the proper component within itself.

object->Attach( Track &thisTrack, int CompId ) {
switch ( CompId ) …
}

Each object has a finite amount of components so your switch cases are not gigantic and sprawling.

The ComponentId indicates that this track is meant to animate a float representing TRANSLATE.X etc.  On a camera you would handle tracks meant for field-of-view, aperture, etc.

A light would accept tracks with ComponentId’s for light range, falloff, specular.r, specular.g, etc.

L. Spiro

Share on other sites

I'm not using the FBX SDK, but it's just because I had already made a FBX Loader when I found out about the SDK... I'm reading the ASCII FBX file, but my 3DS Max is quite old (2010) so the format might've changed a bit, or because it's not the SDK there's some differences on what you've said, but it's nothing troublesome to adapt (like having zeros in all translates/rotates channels, on each keyframe that there is another value change anywhere, even if my animation didn't use that channel).

After I found out about the SDK, since I don't plan on using the FBX files (I'll read the data and save in a format of my game files), I didn't implemented it and kept using the old loader I had.

There doesn’t need to be an association between tracks and joints (or any other object that could be animated) at all.
The name tree simply allows to look up the target object and then the switch case allows to find which component the track will modify on that object, and once the track has a reference or pointer to that component you are done. No other associations need to exist.

That's a great concept, I didn't even had "names" in my scene (I was just going with everything by IDs, names were purely conventional and not necessary), and with this I think it'll become easier to have an animation of "Human Skeleton" and apply it to different humanoid skeletons as long as they have some/most bones names.

I've implemented everything and it looks great so far, but I just realized I'm back to a problem I was having when I posted this, which is the Update function, and I was trying to come up with a structure/system that didn't look as bad as this (current):

class PlayTrack
{
public:
Track* track;
float* valueReference;
};

class PlayState
{
public:
void PlaySequence(wstring sequenceName);

void Update(double updateTime);
private:
Animation* _animation;

vector<PlayTrack*> _tracks; //track instances, with the value refs assigned
vector<Sequence*> _sequences; //an animation sequence defined by the code
vector<double> _sequenceTimes; //the time of each sequence, I didn't make another class so I don't call "new" during the update loop
};

//In my game's main code, it looks like this:
void MyGame::Start()
{

//Creating a new sequence of animation from frames 5 to 10 and naming it WALK
//I'm using frames as time value, from the FBX I'm getting it's always set at 30 FPS but I'll convert it here depending on my update timer

Model* model = Resources.Load_Model(L"MyModel", L"MyAnimationName"); //and some other params

//Here I'm linking this model's animation instance (PlayState) to the object it's supposed to animate
//in this case, the model's skeleton

//It'll start playing an animation
model->PlaySequence(L"WALK"); //will pass this to it's PlayState
}

void PlayState::PlaySequence(wstring& sequenceName)
{
//Getting the sequence from the original animation, so all models share the same sequences
Sequence* sequence = _animation->GetSequence(sequenceName);
if(sequence != 0)
{
_sequences.push_back(sequence);
_sequenceTimes.push_back( (double) sequence->frameStart );
}

//So I'll have for now a "WALK" only animation running, starting at frame 5, going till 10
}

//Everything's set up already, now the engine call the scene to update the animations with a double updateTime
//from the fixed time step impl., where I convert the frames to whatever my fixed time step update rate is
void PlayState::Update(double updateTime)
{
//This is where I'm not happy with how it is

//Updating all times for the playing sequences, this is a must
for(unsigned int i = 0; i < _sequenceTimes.size(); i++)
{
_sequenceTimes[i] += updateTime;
}

//Now for each sequence playing...
for(unsigned int i = 0; i < _sequences.size(); i++)
{
//Check all tracks
for(unsigned int j = 0; j < _tracks.size(); j++)
{
double currentTime = _sequenceTimes[i];

KeyFrame* prevFrame = 0;
KeyFrame* nextFrame = _tracks[j]->track->keyFrames[0];

//Check all it's keyframes
for(unsigned int k = 1; k < _tracks[j]->track->keyFrames.size(); k++)
{
prevFrame = nextFrame;
nextFrame = _tracks[j]->track->keyFrames[k];

if(currentTime >= prevFrame->keyFrame && currentTime <= nextFrame->keyFrame)
{
//Here I got the keyFrame pair I'm looking for
break;
}
}
}
}
}


I can't really save the current time and a pair of keyframes per "PlayTrack" (trackInstance) because there might be more than one sequence playing, so I'm having to loop through so many vectors... isn't there a more elegant (ie. smarter) solution than this?

I'm thinking in maybe moving the sequences elsewhere, and making each "PlayState" able to play a single sequence, but that wouldn't change much since all I'd be doing is moving the sequence loop to another place and copying the PlayState multiple times because of it.

Edit: More Code

Maybe I'm worried for nothing and this might not be a big issue but I can't help but worry "am I doing this right?" and if this is the only way it would be reassuring to listen so I don't think too much over it.

Edited by Danicco

Share on other sites

Okay, hopefully, my last post in this topic!

Finally got it to work perfectly! I had some trouble with bone's origin of rotation since I was using quaternions and had to adjust a few things...

In the end, I did:

class Skeleton
{
public:
//bunch of controls
private:
float _bonePositions[MAX_BONES * 3]; //Bone Local Position
float _boneTranslations[MAX_BONES * 3]; //Bone Translate
float _boneRotations[MAX_BONES * 4]; //Bone Rotation in Quat
}


The local position mostly never changes, but I had to use it to figure the point of each bone to center the rotation around it... and the shader:

#version 330 core
layout(location = 0) in vec3 vertexPosition;
layout(location = 1) in vec2 vertexUV;
layout(location = 2) in vec3 vertexNormal;
layout(location = 3) in vec4 boneIndexes;
layout(location = 4) in vec4 boneWeights;

out vec2 UV;

uniform mat4 transformationMatrix;

uniform vec3 bonePosition[64];
uniform vec3 boneTranslation[64];
uniform vec4 boneRotation[64];

void skinVertex(in vec3 vertexPosition, in vec3 bonePosition, in vec3 boneTranslation, in vec4 boneRotation, in float boneWeight, out vec3 updatedVertexPosition)
{
vec3 rotationOrigin = vertexPosition - bonePosition;

vec3 cross;
cross.x = (boneRotation.y * rotationOrigin.z - rotationOrigin.y * boneRotation.z) * 2;
cross.y = (boneRotation.z * rotationOrigin.x - rotationOrigin.z * boneRotation.x) * 2;
cross.z = (boneRotation.x * rotationOrigin.y - rotationOrigin.x * boneRotation.y) * 2;

vec3 crossQuat;
crossQuat.x = boneRotation.y * cross.z - cross.y * boneRotation.z;
crossQuat.y = boneRotation.z * cross.x - cross.z * boneRotation.x;
crossQuat.z = boneRotation.x * cross.y - cross.x * boneRotation.y;

cross.x = cross.x * boneRotation.w;
cross.y = cross.y * boneRotation.w;
cross.z = cross.z * boneRotation.w;

updatedVertexPosition = vertexPosition + ((cross + crossQuat + boneTranslation) * boneWeight);
}

void main()
{
vec3 skinnedVertex = vertexPosition;

skinVertex(skinnedVertex, bonePosition[int(boneIndexes.x)], boneTranslation[int(boneIndexes.x)], boneRotation[int(boneIndexes.x)], boneWeights.x, skinnedVertex);
skinVertex(skinnedVertex, bonePosition[int(boneIndexes.y)], boneTranslation[int(boneIndexes.y)], boneRotation[int(boneIndexes.y)], boneWeights.y, skinnedVertex);
skinVertex(skinnedVertex, bonePosition[int(boneIndexes.z)], boneTranslation[int(boneIndexes.z)], boneRotation[int(boneIndexes.z)], boneWeights.z, skinnedVertex);
skinVertex(skinnedVertex, bonePosition[int(boneIndexes.w)], boneTranslation[int(boneIndexes.w)], boneRotation[int(boneIndexes.w)], boneWeights.w, skinnedVertex);

gl_Position = transformationMatrix * vec4(skinnedVertex, 1);

UV = vertexUV;
}


And it's working!

Many many many thanks to everyone, that's been a huge help!