Animation takes up too much memory

Eliminate redundant key frames from the animation data. This operation is trivial and can eliminate 90% of key frames in many cases where animation data is densely packed (such as those captured via motion capture).

Each character instance needs its own copy of its skeleton and a time within the animation (plural when blending), not a whole new copy of the animation data itself, which should be shared when possible (obviously not possible when characters each need completely unique animation data).


L. Spiro

In addition to L. Spiro's advise, exporters often have a setting for sampling rates. Higher sampling rates often results in higher fidelity during playback vs how it looks in the modelling application (specifically important when doing IK or other procedural animation); while lower sampling rates use far less memory.

I was wrong. I have looked at the memory profile and it says that the animation data is not copied for each character instance as I thought.
However, the game still uses over 200 mb of memory for animations alone, and I would like to reduce that if possible.

When we were setting up the pipeline, we found that there was only one specific way to get the animations to play properly in the game. It involved baking all the keyframes. I am not sure if we will be able to tweak the export procedure very much without it breaking.

It involved baking all the keyframes. I am not sure if we will be able to tweak the export procedure very much without it breaking.

What do mean by "baking" all the keyframes? And do you mean "keyframes" or just time-indexed data?

From your comments, it sounds like the animation code is broken. You could revise the runtime animation calculations and greatly reduce the amount of animation data needed. For instance, a simple animation for walking would require only 3 keyframes per bone. The vast majority of animation data needs only rotation data (i.e., no scaling or translation). If that's stored as a quaternion, that's 4 float values. Say 30 bones, 3 quaternions each, 4 floats per quaternion, 4 bytes per float value = 1.5K per animation, with the possibility that could be shared among any characters using the same frame hierarchy.

As L. Spiro mentions, eliminate redundant frame data from the data you're importing (not at runtime). If you have 20 frames of animation data that are equivalent (or close enough) to LERP'ing or SLERP'ing between frames 0 and 19, you only need 2 keyframes per bone. If that means revising the way you generate animations, the way you import data into your app, and the way you implement animations at run-time, then you'll have to make a choice.

It sounds like this component you use might just be a bit wasteful with memory.

Is there any reason you dont just do animations yourself? It should only take a few hours to figure it out and put something together. Think of sequences of animations as simple instructions, like in turtle graphics and logo, and see if you can come up with something. It'll be worth it!

You could write some simple routines to interpolate movement of parts of your sprites, or of the joints in a 3D skeletal animation, for 2D split the sprites up into discrete bits, e.g. arms, legs, weapons, faces, eyes etc. You can then move and animate only the moving elements.

I am looking at the code that pre-compiles the animation data coming from the exporter.

It looks like it is storing 60 keys per second, regardless of what it receives from the exporter.

Here is the code:

                // Step through time until the time passes the animation duration
                while (time <= animationDuration)
                {
                    AnimationKeyframe keyframe;
                    // Clamp the time to the duration of the animation and make this 
                    // keyframe equal to the last animation frame.
                    if (time >= animationDuration)
                    {
                        time = animationDuration;
                        keyframe = new AnimationKeyframe(new TimeSpan(time),
                            channel[channel.Count - 1].Transform);
                    }
                    else
                    {
                        // If the channel only has one keyframe, set the transform for the current time
                        // to that keyframes transform
                        if (channel.Count == 1 || time < channel[0].Time.Ticks)
                        {
                            keyframe = new AnimationKeyframe(new TimeSpan(time), channel[0].Transform);
                        }
                        // If the current track duration is less than the animation duration,
                        // use the last transform in the track once the time surpasses the duration
                        else if (channel[channel.Count - 1].Time.Ticks <= time)
                        {
                            keyframe = new AnimationKeyframe(new TimeSpan(time), channel[channel.Count - 1].Transform);
                        }
                        else // proceed as normal
                        {
                            // Go to the next frame that is less than the current time
                            while (channel[currentFrame + 1].Time.Ticks < time)
                            {
                                currentFrame++;
                            }
                            // Numerator of the interpolation factor
                            double interpNumerator = (double)(time - channel[currentFrame].Time.Ticks);
                            // Denominator of the interpolation factor
                            double interpDenom = (double)(channel[currentFrame + 1].Time.Ticks - channel[currentFrame].Time.Ticks);
                            // The interpolation factor, or amount to interpolate between the current
                            // and next frame
                            double interpAmount = interpNumerator / interpDenom;
                            
                            // If the frames are roughly 60 frames per second apart, use linear interpolation
                            if (channel[currentFrame + 1].Time.Ticks - channel[currentFrame].Time.Ticks
                                <= ContentUtil.TICKS_PER_60FPS * 1.05)
                            {
                              //  context.Logger.LogImportantMessage("Lerp between frames {0} and {1}, interpAmount: {2}", currentFrame.ToString(), (currentFrame + 1).ToString(), interpAmount.ToString()); //  input.Duration.Ticks.ToString(), ((long)(durationFactor * input.Duration.Ticks)).ToString());

                                keyframe = new AnimationKeyframe(new TimeSpan(time),
                                    Matrix.Lerp(
                                    channel[currentFrame].Transform,
                                    channel[currentFrame + 1].Transform,
                                    (float)interpAmount));
                            }
                            else // else if the transforms between the current frame and the next aren't identical
                                 // decompose the matrix and interpolate the rotation separately
                                if (channel[currentFrame].Transform != channel[currentFrame + 1].Transform)
                            {
                            //    context.Logger.LogImportantMessage("Slerp between frames {0} and {1}, interpAmount: {2}", currentFrame.ToString(), (currentFrame + 1).ToString(), interpAmount.ToString()); //  input.Duration.Ticks.ToString(), ((long)(durationFactor * input.Duration.Ticks)).ToString());

                                keyframe = new AnimationKeyframe(new TimeSpan(time),
                                    ContentUtil.SlerpMatrix(
                                    channel[currentFrame].Transform,
                                    channel[currentFrame + 1].Transform,
                                    (float)interpAmount));
                            }
                            else // Else the adjacent frames have identical transforms and we can use
                                    // the current frames transform for the current keyframe.
                            {
                                keyframe = new AnimationKeyframe(new TimeSpan(time),
                                    channel[currentFrame].Transform);
                            }

                        }
                    }
                    // Add the interpolated keyframe to the new channel.
                    outChannel.Add(keyframe);
                    // Step the time forward by 1/60th of a second
                    time += ContentUtil.TICKS_PER_60FPS;
                }

But at runtime, interpolation is performed again. So as some of you mentioned, I may be able to reduce this data and still get smooth playback.

As a quick hack, I'd setup that the animation to take just 10% of the time and play it 10 times slower. If that reduces the size and still looks good, then there is really a sampling issue with the exporter as you're assuming.

As a quick hack, I'd setup that the animation to take just 10% of the time and play it 10 times slower. If that reduces the size and still looks good, then there is really a sampling issue with the exporter as you're assuming.

Yes, but the code i posted above suggests that everything that gets exported is overridden in a resampling loop.

else // Else the adjacent frames have identical transforms and we can use
// the current frames transform for the current keyframe.
{
keyframe = new AnimationKeyframe(new TimeSpan(time),
channel[currentFrame].Transform);
}

Here, I might as well not store any keyframe at all. I wonder how much data could be saved that way.

I may be able to reduce this data and still get smooth playback.

If your code (properly) interpolates animation data at runtime, do you need to do anything other than just load the data? I.e., game assets in general should be "ready to load" at runtime.

I may be able to reduce this data and still get smooth playback.

If your code (properly) interpolates animation data at runtime, do you need to do anything other than just load the data? I.e., game assets in general should be "ready to load" at runtime.

The code I posted only runs at build time. It creates the key frame arrays that the game interpolates between during runtime.
My problem was that these arrays were so huge.

I reduced the sampling rate to 30 frames per second from 60. The memory used went down from 206 mb to 108 mb, while animation quality looks the same.

I will try to reduce it a bit further, then I will probably try removing the redundant key frames as was suggested.

I have eliminated the redundant key frames (frames that have the same transform matrix as the preceding one) and have now reduced the size to 84 mb.

I am wondering if there are more low-hanging fruits that can bring it down further....

The vast majority of animation data needs only rotation data (i.e., no scaling or translation). If that's stored as a quaternion, that's 4 float values. Say 30 bones, 3 quaternions each, 4 floats per quaternion, 4 bytes per float value = 1.5K per animation, with the possibility that could be shared among any characters using the same frame hierarchy.

Yes, what I have is 4*4 matrices, likely quaternions. If I decompose them into vectors, I would need to change the runtime animation code to use vectors instead... I am not sure if I want to do that at this point...

This is what the struct looks like:

 /// <summary>
    /// Represents a keyframe in an animation track.
    /// </summary>
    public struct BoneKeyframe
    {
        /// <summary>
        /// Creats a new BoneKeyframe.
        /// </summary>
        /// <param name="transform">The transform for the keyframe.</param>
        /// <param name="time">The time in ticks for the keyframe.</param>
        public BoneKeyframe(Matrix transform, long time)
        {
            this.Transform = transform;
            this.Time = time;
        }
        /// <summary>
        /// The transform for the keyframe.
        /// </summary>
        public readonly Matrix Transform;
        /// <summary>
        /// The time for the keyframe.
        /// </summary>
        public readonly long Time;

    }

Most of the keyframes are just hanging around in memory. How could I compress it until the transform is needed..?

As L. Spiro mentions, eliminate redundant frame data from the data you're importing (not at runtime).
If you have 20 frames of animation data that are equivalent (or close enough) to LERP'ing or SLERP'ing between frames 0 and 19,
you only need 2 keyframes per bone.

I have already eliminated the frames that are equal. But is the situation really that common in animation that a handmade animation sequence is similar to a SLERP? How would I detect this case. How do I select the endpoints in the animation sequence to compare, when doing the loop I posted above?

I found some useful information here:

http://www.digitalrune.com/Support/Blog/tabid/719/EntryId/131/Character-Animation-Compression.aspx

I have eliminated the redundant key frames (frames that have the same transform matrix as the preceding one) and have now reduced the size to 84 mb.

This isn't removing the redundant keyframes.It means that you have to remove frames that can be represented by using it's 2 neighbors(aka the derivative in moment t is the same as in t-1).

is the situation really that common in animation that a handmade animation sequence is similar to a SLERP?

Yes, if the person creating the animation uses rotations between keyframes as the basis for the animation. I.e., to create a cyclic animation (walk, run, raise/lower arm...), a typical starting point for the animation is (e.g. arm raise/lower): frame 0 - arm down, frame t - same as frame 0, frame t/2 - arm raised. SLERP from frame 0 to frame t/2, SLERP from frame t/2 to frame t.

"Redundant frames" needn't mean only frames that are the same. A redundant frame is any frame that is sufficiently close to the value you would calculate by interpolating/SLERPing the previous and following frame. I.e., if you're interpolating frames anyway, and the frame data for time t is what you get from interpolating frame t-1 and frame t+2, frame t is redundant.

Re: redundant frames - ninja'd

EDIT2:

redundant key frames

That's oxymoronic. A keyframe, by definition, is a key frame. When you eliminate all the frame data that isn't necessary for the desired animation, what's left are keyframes.

Alright then. But L. Spiro used the same term...


I am trying to replace this, inside BoneKeyFrame:

/// <summary>
/// The transform for the keyframe.
/// </summary>
public readonly Matrix Transform;

with a custom class/struct. It should contain an optional translation vector, optional scale vector (when different from 1) and a rotation quaternion.
How can I design it so it becomes as compact as possible... The point is to reduce the size of the BoneKeyFrame array.

But Spiro used the same term...

Yeah, the terms frame and keyframe are often used interchangeably. In the context of data reduction, it's a pedantic observation - thus the "wink"