# Motor control redux

This topic is 756 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

This is a continuation of my Motor control for physics-based character movement thread from last year.

I've made some progress since last year! The Dood can remain standing now, given a narrow range of initial conditions!

Video just for fun:

(Also feel free to check out any of the other videos on my channel.)

My Current Algorithm

Character configuration:

When I talk about the 9 bones of the lower body, I am referring to: ltoe, lheel, llleg, luleg, pelvis, ruleg, rlleg, rheel, and rtoe.

When I talk about the 4 foot bones, I am referring to: ltoe, lheel, rtoe, rheel.

Each joint has a min/max rotation and min/max torque on each of 3 axes, specified relative to the "parent" bone. For many joints the axes are the same as the local coordinate axes. A notable exception is the knees, whose primary axis of rotation is not perfectly horizontal (because the legs in the reference pose (shown above) are not perfectly vertical); the knees have a decent range of rotation on this primary axis, and next to no range of rotation on their other two axes.

Offline optimization parameters:

• For each bone of the lower body, a float describing the importance of matching that bone's desired torque
• For each bone of the lower body, a 3x3 matrix mapping a desired net force on the Dood to an offset to that bone's desired orientation
• For the upper torso bone, ditto, and another 3x3 matrix mapping CoM error to the same
• For each of the 4 foot bones, a float defining what fraction of the applied joint torque is expected to take, and what fraction of the measured ground torque is expected to be unchanged from the previous tick

Offline optimization process:

• Initialize a population of 100 candidates. The first 10 have the average of min and max values for all parameters; the rest are this but with random mutations. (Or load the population from a file)
• Generate 100 random subtests. Subtests have variables like: a random initial linear velocity to add to the upper body; an amount to move the desired yaw & pitch by over the course of the simulation
• Repeat until the user says to stop:
• Go through the 100 candidates and find the top 10, by total of the 12 scoring categories (see below):
• For each subtest:
• Simulate for 5 seconds (5x60 = 300 ticks)
• For the first 11 scoring categories (9x bone orientation error squared, 1 CoM error, 1 energy cost), integrate: 1 - e-k * value dt (so this will always range from 0 to 5)
• If a failure condition (either foot is completely off the ground, or the CoM deviates from its initial pos by more than a certain radius) for more than 2 consecutive ticks, the subtest is failed early (stop simulating/integrating). The 12th category score is assigned a value = the amount of remaining time * the number of "normal" categories (11) ... thus the total is as if the candidate had gotten the worst possible value in each category for the remainder of the simulation.
• So the total, and the 12th category score, will both always range from 0 to 55.
• If the top 10 for this generation have already been found and the total score of this candidate is worse than the worst of the top 10, the candidate fails early.
• The next generation will include the top 10 candidates from this generation, and 2 crossovers with random mutations for each pair therebetween

At runtime:

Let's call this the "get me there immediately" formula (GMTI for short):

Force = (((desired pos) - (current pos)) / timestep - (current vel)) / timestep * mass

For the bones of the upper body, I use GMTI to come up with desired bone torques, and add/subtract them in a sequence along the joints to come up with joint torques... this results in all of the 'junk' torque accumulating in one place, currently the upper torso. This works. It could probably be improved by taking linear momentum into account, but it works.

Let W be the bone weight vector (from the offline optimization parameters)
Let b be the desired torques for each bone.
Let f be the torques applied by the joints' "motors".
Let A be a matrix which maps joint torques to the predicted bone torque effects thereof.

I minimize ||Af - b||2W (i.e. the weighted sum of the squares), subject to the constraint that f must be within some AABB.

The vector b is computed as...

Use GMTI(CoM --> initial CoM) to compute a desired net force on the Dood

Each bone has a desired orientation. Currently I'm using the initial orientation as the desired orientation
Offset the bone's desired orientation by the desired net force times the corresponding matrix
Use GMTI(bone orientation --> bone's initial orientation) to compute a desired torque on that bone

For the 4 foot bones, subtract the measured foot-ground torque times the corresponding scalar from the offline optimization parameters

Matrix A is computed as...

Joint torques are specified in the joint's local coords (to make the bounds on them be a convenient AABB).
Bone torques are specified in world coords.
For 3x3 submatrices mapping a joint torque to the torque on an adjacent bone, the values are either + or - of the joint's rotation matrix.

For torques affecting the 4 foot bones, scale the relevant 3x3 submatrices by the corresponding scalar from the offline optimization parameters.

So now what?

This works for standing (given a narrow range of initial conditions), but I have no idea what to do from here.

Any assistance is appreciated.

Edited by Aken H Bosch

##### Share on other sites

Ok, now I'm working on something more specific: I'm trying to account for linear. So now instead of my A matrix mapping joint torques to bone torques, it maps joint torques to bone torques and bone forces, and the array b now contains the desired bone torques and the desired bone forces.

Except that for b I also have to subtract the expected "default" bone forces, i.e. the forces to expect when no joint torques are applied.

I don't know how to compute that.

##### Share on other sites

Ok, new goal. Even more specific:

http://grail.cs.washington.edu/projects/loco/loco_paper.pdf

This paper defines ?, the joint torques (the quantity I'm ultimately trying to compute) in terms of f, the forces between action/reaction frames which will arise from applying those torques, using the formula JTf = ?. The math works, but the computation works backwards vs the underlying causation. I'm interested in finding a way to instead express f as a function of ?. Obvious problem being that you can't invert a non-square matrix. But there's also a constraint Zf = 0 which I think will make it solvable...

J is a matrix such that JT times a force vector gives a torque vector. So let's say J is F rows by T columns

Z is a matrix such that Z times a force vector gives the zero vector of some arbitrary length. So let's say Z is N rows by F columns.

Z is defined in the paper as "a matrix whose columns are the basis for the null space of JT" ... I implemented the computation of Z a while ago, but I've forgotten a lot of how it worked. But I'm pretty sure this means N has to be less than or equal to T.

(As I'm writing this I'm constantly having to remind myself whether FxT means F rows by T columns, or F columns by T rows. (It's the first one))

I'm hoping it's mathematically possible to find a matrix D such that D? = f ... meaning it would be F rows by T columns.

The paper's constraint Zf = 0 is thus equivalent to ZD? = 0. Therefore D must satisfy ZD = 0.

And the definition JTf = ? is equivalent to JTD? = ?. Therefore D must satisfy JTD = I.

I'm thinking if all I had to go by was that ZD = 0 and JTD = I, it would be under-constrained, and that somehow the solvability relies on the knowledge that Z's columns are the basis for the null space of JT.

Is it possible to find such a matrix? If so, how?

##### Share on other sites

Little help?

Is this stuff too advanced for everyone here?

##### Share on other sites

Maybe you can simplify your description of the problem, if possible with some diagrams

I know your second post attempts to do that, but you could try again with some diagram. The way the brain work, while trying to break the problem down for others to understand you might have an epiphany and the answer pops out to you. That has happened to me in the past

##### Share on other sites

Illustrations... good idea. Not really sure what I can illustrate, but I've tried adding a section "Character configuration" to the OP showing the character's bones and the joints connecting them. And explaining what I mean when I talk about "the 9 lower body bones" or "the 4 foot bones".

Unfortunately I can't take any new screenshots or record any new video, because my dev computer died on me. I've got stuff backed up, but won't be able to run my code until I get a new computer. (The hunk-of-junk I'm posting this from doesn't even support GLSL)

Recent changes:

I've made some changes to my algorithm recently (before my computer went kaput). It now takes into account the linear dynamics of the system, not just the angular! Sort of.

The offline optimization parameters are now:

struct FrameParams
{
float qp_weights_angular[NUM_LOWER_BODY_BONES];
float qp_weights_linear[NUM_LOWER_BODY_BONES];
float xprod_frac[NUM_LOWER_BODY_BONES * NUM_LOWER_BODY_BONES];
};


(approximately)

The floats qp_weights_* are weights for the Quadratic Program (the minimization of ||Af - b||2W, as seen in the OP). In addition to each bone of the lower body having a desired torque, they also now have desired forces. Except that instead of having a desired pelvis force, I have a desired net force on the pelvis, torso1, torso2, head, lshoulder, and rshoulder. The values for these weights can range from 0.25 to 4.0, except for the angular weights corresponding to luleg, llleg, ruleg, and rlleg, which are always zero.

For each bone of the lower body, I take the cross product of the angular velocity of that bone (including the effect of any joint torques on that angular velocity) and the corresponding vector from xprod_radii to come up with a predicted force for that bone, then use the scalars in xprod_frac to predict how that force will affect all of the other bones. I don't really have a model for how this should work, but the GA is able to come up with values that let the Dood stand ... given a narrow range of initial conditions.

Also, I've ditched the part where each component of the score is always between 0 and 1 (times the sim duration) ... a while ago actually. Now I just use the weighted sum of a bunch of categories of errors-squared.

I'm still at a loss for where to go from here.

If there's anything else I can do to improve my explanation of the problem, please let me know.

##### Share on other sites

If I make a few reasonable-ish approximations, I can reduce the problem of "determine the bone forces based on the bone torques" to a geometry problem, then take derivatives.

The approximations:

1. The angular velocities going into the constraint solver (let's call this ?) will be the same as the angular velocities coming out of the constraint solver (?').

2. One or both feet are planted on the ground, therefore the pos/ori of those bones are constrained.

3. All of the joints stay in their sockets.

What that gets me:

From a proposed set of joint torques ?, I can determine ?. If I approximate ?' as equal to ?, I can go on to use ?' to determine the resulting bone orientations ?. Then I can place the oriented bones socket-to-socket, starting from one of the constrained foot bones' positions, to determine the position x of every bone in the system.

From x I can work backwards to determine v', a and then finally the thing I'm interested in, F.

Rather than actually compute all of that for every proposed ?, however, I intend to compute it once for ? = 0 (or perhaps the value of ? from the previous simulation step?), and use the Jacobian matrix ?F/?? to determine the values of F for other ? in the vicinity of that point.

I'm already foreseeing a few problems:

Firstly, the approximations themselves:

1. The approximation ?' = ? becomes less accurate the more "interesting" the situation gets. Also notice that we're determining v', the linear velocities coming out of the constraint solver, in a manner that doesn't even involve v, the linear velocities going into the constraint solver. I'll need a slightly better approximation of ?', one that takes v into account somehow.

2. The pos/ori of the feet aren't actually constrained. Normal force and friction will help, but that only works when the foot is being pressed downward sufficiently for the friction to overcome any lateral force. And it doesn't work at all if the foot is being pulled upward. Also, if both feet are planted, the geometry is over-constrained! (There's also the fact that there are two bones in each foot, but I don't expect that will be too difficult to deal with.)

3. The more energy there is in the system, the more residual error will be left over when the constraint solver finishes. Stuff like joints not being quite in their sockets. This is probably trivial compared to approximation 1, though... unless the Dood in question has just been fired out of some sort of giant rifle.

And finally, there's the assumption that it's linear; that F(?) = F(?0) + (? - ?0)(?F/??). In that paper I keep referring to they were able to do something like this, but they're using the Jacobian differently: they do J = ?x/?? (they use x for the relative bone positions (and orientations?) and ? for joint angles), and then they determine their joint torques using the formula JTf = ?, where f is the relative forces (and torques?) between action and reaction frames. (Note that f and ? use the same coordinate systems as x and ?.)

I'm hoping it's pretty close to linear, at least in the areas where it matters. I can't afford to actually calculate all of those bone positions.

Closing question:

How can I improve my approximation of ?', presumably taking into account v somehow, without much additional cost?

(Once I get my new computer (in about two weeks  :(), I'll try to implement what I've just described, and then I'll find out how well it works.)

Edited by Aken H Bosch

##### Share on other sites

Well I implemented it, but it's not working and I don't know what's wrong with it.

Implementation choices:

I ended up using a numerical approximation of the partial derivatives formula from the above post, and letting the GA search come up with the value for dx.

To address the problem being over-constrained (i.e. working from the left foot up to the pelvis, and working from the right foot up to the pelvis, you should arrive at the same location; else the bone orientations chosen were invalid), I tried two different schemes:

1. The mismatch between the predicted positions of the pelvis (left leg prediction and right leg prediction) is a goal property ... like how each bone's desired forces is a goal property: the QP searches for values of the independent variable to minimize total weighted error squared of the dependent variables. The weight for the pelvis mismatch term is determined by the GA (offline optimization). The desired value for this goal property is Vec3(0, 0, 0).
2. The mismatch between the predicted positions is a strict constraint enforced by the QP.

But neither worked.

Null-space control?

I'm thinking maybe it's a problem with null-space control, one which the paper I've been taking a lot of ideas from also had to deal with: that there isn't any value for joint torques that will produce an upward pelvis force when the knees are fully extended. If I can't think of anything else, I'll try to adapt their solution to my setup.

##### Share on other sites

I don't have any real progress to report on this, because I haven't gotten any help from anyone.

But it's coming up on 60 days since the last post in this topic, at which point it will get locked due to inactivity... so I have to post something.

I finally migrated my Google Code repository to GitHub.

More recently, I wrote a Lua interface for the motor control system in the hopes that I could convince some of my friends to try their hand at it, but so far only a few have tried running the program, and AFAIK none have tried to script anything with it.

But the GA and QP features have been removed, and simulation is now done in real time instead of as fast as possible. I'm considering adding the QP back in and exposing it to Lua as just one option for how to control the system.

The Lua interface also adds control over the vectored-thrust jetpack system, which finally has a graphical effect now.

(never mind the bit of elbow poking through the flames)

I need feedback.

Seriously. I can't progress without feedback.

##### Share on other sites

What is another forum I could try asking on that would be more specialized for this kind of topic, and has active members?

1. 1
2. 2
3. 3
4. 4
Rutin
13
5. 5

• 26
• 10
• 9
• 9
• 11
• ### Forum Statistics

• Total Topics
633696
• Total Posts
3013394
×