Sign in to follow this  
JoakimPdelski

bone matrices in 3d formats...

Recommended Posts

I'm making a Blender importer/addon for a game format we're using. The format uses 4x4 inverse transform matrices for bones. I'm very puzzled because when I import the bones to Blender, or load the file in our game engine, the bone rotations seem way off.
Here are some screens:
www.postimage.org/image/258d8jh8k/
www.postimage.org/image/25889xfr8/
www.postimage.org/image/2584yuqro/

I'm sure most of you will agree that these rotations don't make any sense. Its not very likely that the 3d artists made the bones like that. The only explanation is that some 3d modellers store matrices in a different way or something.

I tried to do the same with Blender.
Seems correct:
www.postimage.org/image/252j7aeis/
www.postimage.org/image/2536ct9fo/

(note that the bones aren't connected in any of the images, because the 3d formats dont store real bones per se, only transform matrices and the scale value in the matrix is not used for the lenght of the bone (it's always 1.0)

I then tried to import other formats in Blender/3d engine and I got the same results.

However, if I play the animations created for those 3d models, everything looks exactly as it should. So this means I'm not doing anything wrong during import, because the animations would screw up if the bones had wrong angles.

What does this mean? Why should I care?
This means that I can't make a proper importer for Blender, as those angles of bones would be unusable for modding. It also means if I hand edited the bone angles or just connected the bones, then the newly exported models wouldn't be campatible with the existing animation files.

So I'm pretty stuck. I could post the importer and some sample files, but like I said it's also the case for other formats.
Why is it like this? Is this some kind of conundrum or historycal meaning? Can something be done?

this is the blender function for loading and creating the bones:
[code]
def read_joint(index, file_object):
# 64 byte matrix (16* 4byte float), 4x4, inverse
data_chunk = file_object.read(64)
rawlist = struct.unpack_from('<16f', data_chunk) # '<16f' = "little endian, 16* float (4 byte)"
r0 = [[rawlist[i] for i in range(el,el+4)] for el in range(0, 15, 4)]

# create a Blender matrix
bmat = Blender.Mathutils.Matrix(r0[0],r0[1],r0[2],r0[3])

bmat.invert() # inverts this matrix

# You need to set this explicitly.
skeleton.makeEditable()

newBone = Blender.Armature.Editbone()

newBone.matrix = bmat

armature.update()
[/code]

heres an example skeleton with the weird bone angles, in blend format:
www.megaupload.com/?d=H72YS2Q8

Share this post


Link to post
Share on other sites
I don't know about blender exactly, but in Maya, the modeling program doesn't care how the bones are laid out in the file. The only thing that matters is how you constrain them. That means at work we'll end up with many models where many of the bones point "back" or "down", because it is easiest to just lay out the bones from some ortho projection resulting in many of them "facing" in one direction, and not along the limb. You can always constrain them on any axis you want, so, to the artists, it usually only matters that [i]some[/i] axis points down the length of the limb.

From a gameplay perspective, that means it is impossible to use bones for game data, as they are effectively pointing in random directions. We resort to skinning attachment point matrices placed on the model. That way, the skinned attachment will point in the correct direction. This only means storing a single bone-relative-matrix for each attach point.

Share this post


Link to post
Share on other sites
Might not be entirely relevant, but I know that when I started with quaternions, I had many bone joints with rotation quaternions that were wildly incorrect. It turned out, that the quaternions that were out of the correct range were actually synonymous, and weren't handled properly.

Such as, sin(90 degrees) is the same as sin(450 degrees), but for a function that expects degrees in the range 0 <= x < 360, the answers could be unpredictable for values outside the range. Might not be the problem, but just a thought.

Share this post


Link to post
Share on other sites
First, thank you for replying.

Now then, when you say you had seemingly wrong rotations, did the animations still work as expected?
When you say "synonymous" does that mean they can be replaced with each other and not cause any problems at all?

I dont set the rotations as quaternions in Blender or the game engine and I get the same results. However, I dont know what the functions do internaly, so maybe they set them as quat in the end.

Share this post


Link to post
Share on other sites
[quote name='joanp1' timestamp='1302952118' post='4799100']
i don't think that answers my question, or maybe you didnt get what I mean?
[/quote]
hmmm. Maybe I didn't understand what you meant.
You have some models with what you think are invalid bone directions. Do they animate properly still? or are the bones actually messed up, and you can't get the model to skin properly because of it?

What I was getting at in my other post, if I open up one of our humanoid models here at work, we have them facing down positive Z. Looking at the bones, the "Z" or "at" of almost every single bone is pointing left-ish. The "X" or "left" of every bone is point up-ish on the right side of the model, and that is mirrored down-ish on the left side of the model. Now, if I go to some other humanoid model (still facing down +Z), all the bones are facing forward-ish, with X pointing up or down based on the side of the model. So, depending on what direction the limb is facing in the file, it wil have some axis aligned to it, but its just as likely to be any axis, positive or negative.
If you are drawing those arrow markers based solely on the bone's matrix from the exporter, then it could technically be facing anywhere and still be correct as long as SOME axis is aligned along the pivot the artist needed for that joint.

If you look at your [url="http://www.postimage.org/image/25889xfr8/"]second image[/url], if that were in standard bind pose, almost all the bones would be facing backwards-ish or up-ish. This is almost exactly what I see in our models at work.

Share this post


Link to post
Share on other sites
[quote]You have some models with what you think are invalid bone directions. Do they animate properly still? or are the bones actually messed up, and you can't get the model to skin properly because of it?[/quote]
Well, the angles seem weird, but they animate the model properly [b]in the game engine[/b]. The animation transform matrices aren't relative to the bone transform matrix, but the vertices are, I guess. If you set the bone matrices to something insane in the modeller (or directly in the 3d file), you will notice when rendering the bones, that the bones are in place during animation, but vertices are not. I think this will be also the case if modifying only the angles.

My goal right now is to make a Blender importer for the 3d format of the game, and although I didn't have to care about how the bones looked in the game engine and it worked as long as the model animated correctly, I really need to have proper bones after import in the 3d modeller. You have to agree that it would be inconvenient to work with such bones. They need to have proper angles and be connected (I can connect after having proper angles easily). If I hand edit the bone angles in the importer, then it wont be campatible with the existing animations anymore, right? Even if it was, you can't know what bone doesnt seem right beforehand for every single model.

[quote]What I was getting at in my other post, if I open up one of our humanoid models here at work, we have them facing down positive Z. Looking at the bones, the "Z" or "at" of almost every single bone is pointing left-ish. The "X" or "left" of every bone is point up-ish on the right side of the model, and that is mirrored down-ish on the left side of the model.[/quote]
You open what? In your 3d modeller? Is it the original model file, or in the game's format? Most of what you described is also like that for me. Is there an explanation for that? Is it caused by the matrix data in the 3d modellers during export?
[quote]
If you are drawing those arrow markers based solely on the bone's matrix from the exporter, then it could technically be facing anywhere and still be correct as long as SOME axis is aligned along the pivot the artist needed for that joint.[/quote]
I don't understan this.
It also looks like that when creating bones in Blender with our model importer, not just when rendering the joints with those "markers" in the game.
[quote]
If you look at your second image, if that were in standard bind pose, almost all the bones would be facing backwards-ish or up-ish. This is almost exactly what I see in our models at work. [/quote]
I don't understand this too (btw bind pose is "rest pose"?)

Share this post


Link to post
Share on other sites
[quote name='joanp1' timestamp='1303036751' post='4799433']
Well, the angles seem weird, but they animate the model properly [b]in the game engine[/b].
[/quote]
Well, then the angles are probably fine.

[quote]
You open what? In your 3d modeller? Is it the original model file, or in the game's format?
[/quote]
Open the original model in the original modeling program. Or, draw the skeleton in our game engine. Its the same data on both sides, I'd see the same thing.

[quote]
Most of what you described is also like that for me. Is there an explanation for that? Is it caused by the matrix data in the 3d modellers during export?
[/quote]
No. This has nothing to do with importing or exporting anything. When the artists place a bone, in Maya atleast, it faces along the direction they click (either twards or away from). So, from an ortho perspective (top, front, side), placing all the bones would result in them all facing one direction. Once they've placed some bones, they'll rotate and tweak them while they assign constraints and controllers. They rotate stuff till some axis properly represents the joint, but that axis isn't the same for any particular limb.

[quote][quote]
If you look at your second image, if that were in standard bind pose, almost all the bones would be facing backwards-ish or up-ish. This is almost exactly what I see in our models at work. [/quote]
I don't understand this too (btw bind pose is "rest pose"?)
[/quote]
I've always heard of it as "bind pose", and tones of google hits show models in what i meant by "standard bind pose". The human figure making a cross shape with his body straight up and down, legs slightly spread and arms outstretched.


Animators don't use bones to animate. And the bone's don't entierly describe the skeleton in a modeling program. That might be part of your confusion. While the bones alone are enough for a game engine, that just has to skin the verts, the two other things your model importer needs, if available, is a way to import the constraints and controllers. Constraints limit rotation, length, and scale of a particular bone. You can also have IK constraints that the modeling program uses to solve where intermediate joints need to move to, so moving a hand around automatically animates the elbo and shoulder joints. You also have controllers, where the person who rigged the model can specify that "fist controller A, goes 0->1", where 0 corosponds to an open hand, and 1 corosponds to all the proper rotations to make a fist. So, animators can just slide that one controller to modify how open/closed the hand is, without dealing with the ~11 bones in an articulated hand.

Share this post


Link to post
Share on other sites
[quote]Well, then the angles are probably fine.[/quote]
Which brings me to my original question: what can be done to have better rotations without making the new models unusable with the existing animations? I should have asked this earlier: Do bind pose rotations matter (are the animations relative to it)? I know positions do. If I manually rotated the bones in the negative x axis to match the ones in the other side, would the new bones play the old animations correctly?

[quote]Open the original model in the original modeling program. Or, draw the skeleton in our game engine. Its the same data on both sides, I'd see the same thing.[/quote]
I are pretty lucky then.
But are you saying you don't have the bones connected in your modeller?
Could you post some demo picture?
[quote]
Animators don't use bones to animate. And the bone's don't entierly describe the skeleton in a modeling program. That might be part of your confusion. While the bones alone are enough for a game engine...[/code]
The format only uses skeletal animation.

Share this post


Link to post
Share on other sites
[quote]
what can be done to have better rotations without making the new models unusable with the existing animations?
[/quote]
You'd have to fix all the animations if you change the initial bone rotations. Since we do everything in Maya at work, we'd have to make a new tool to do what you are talking about, since rotating the bone in the modeling program instantly screws up all the animations.

[quote]
But are you saying you don't have the bones connected in your modeller?
Could you post some demo picture?
[/quote]

As you can see, almost every single bone points to the right of the image. A few in the face point forward. The rig is fairly uniform, and the left axis of most the bones points along the direction of the bone, but, that is mirrored in spots and points up or down the bone connection. And again, some in the face area don't have anything pointed in any particular direction.

Everything is connected, but that isn't achieved through the bone matrix. That is based on how they are parented in the hierarchy. Maybe you aren't putting each bone under its proper parent when you import the skeleton?
[img]http://tsanchez.is-a-geek.com/media/bonedemo.jpg[/img]

Share this post


Link to post
Share on other sites
Wait, youre saying the angles bones get when connected to their parent are not stored in the bone matrices? If its true then [b]that's great news and everything makes sense now![/b] I have asked so many places and nobody could hint me that. You see, I was afraid that by connecting the bones, I would ultimately change their (rotation and scale part of the) transform matrices and thus make them unusable with the existing animations. And that is why I thought that the bone rotations were imported wrongly, because they didnt have the angles they would before when exported from the modeller. However, Im afraid that this might not be true, because even then how would the outmost bones have correct rotations? Their tips aren't connected to anything. In your pic it seems at least that outmost finger bones are facing the same direction on both sides, although their matrices are different...

Share this post


Link to post
Share on other sites
It is exactly, as KulSeran said - the game model does not have original bone directions and lengths coded in, but only local joint coordinate systems. I have had similar problem with importing/exporting myself - it can be solved with some pain, though...
Basically, what you need, is to add to each joint matrix two parameters:

[list][*]A matrix describing the bone orientation[*]Bone length[/list]
These can be done semi-automatically for standard meshes:

If the bone has only one child, construct lookAt matrix from this joint to child origin (and multiply it with some matrix, that aligns correct axis - for Blender i think it is Y). The length is simply the distance from current joint to child
If the bone has many children, take the average of child origins and construct lookAt matrix from this joint to the average point (and multiply with alignment matrix). the length is the distance from current joint to average point. This works for things like backbone and palms, but gives probably wrong results, if there are extra bones.

This new bone direction matrix has to be post-multiplied to bone matrixes in importer. Also, in your importer save the original joint directions (or keep the original model somewhere). Connect bones in importer by parent/child information.

Now user can modify model in Blender. If some bone directions are guessed wrong, they can also be adjusted in blender to make the work easier.

If exporting model from Blender, you have to translate the bone matrixes back to the original joint coordinate systems. It may be tricky, if modeller has changed bone lengths - in that case you have to keep the new object-space positions intact and only transform rotations. Otherwise you can simply build the original format model from the original skeleton and exported skin from Blender.

If you plan to edit animation in Blender, those have to be converted too, of course.

Share this post


Link to post
Share on other sites
Som am I right, bone matrices aren't changed in the modeller when connecting the bones? "Everything is connected, but that isn't achieved through the bone matrix.".
[quote]Wait, youre saying the angles bones get when connected to their parent are not stored in the bone matrices?[/quote]
can you please answer this?
Because then connecting would work for every bones except for outmost bones with no child bone.

[quote]and multiply it with some matrix, that aligns correct axis[/quote]
What?
If you mean to get the bone lenght, then i could jsut connect the tip of the parent bone with the base of the child bone, that would also solve the rotations...

[quote]Now user can modify model in Blender. If some bone directions are guessed wrong, they can also be adjusted in blender to make the work easier.[/quote]
But thats my question, if I hand edit the bones in Blender, then the new model won't be campatible with existing animations right?

[quote]If exporting model from Blender, you have to translate the bone matrixes back to the original joint coordinate systems. It may be tricky, if modeller has changed bone lengths - in that case you have to keep the new object-space positions intact and only transform rotations. Otherwise you can simply build the original format model from the original skeleton and exported skin from Blender.[/quote]
Not sure if I get most of this, but you ultimately think of hard-coding the bones from the file and ignoring what data was in Blender?

Share this post


Link to post
Share on other sites
[quote name='joanp1' timestamp='1303222679' post='4800369']
Som am I right, bone matrices aren't changed in the modeller when connecting the bones?
[/quote]
That depends, you can set either the world matrix, or the matrix local to the parent. Just choose the one that corosponds to your data when importing and connecting your joints.

[quote]Wait, youre saying the angles bones get when connected to their parent are not stored in the bone matrices?[/quote]
It's not stored in the world-coordinate bone matrix. You can calculate the local-to-parent bone matrix if you want.

[quote]
lauris71, snip....
[/quote]
Lauris71, i think, was trying to explain how you could create a process to convert all the bone matrices in your model and animation files to different matrices while not breaking the rig. It shouldn't be needed for what you are doing.

Share this post


Link to post
Share on other sites
I think there is a confusion here. I see that in your image the transform matrices are like mine, but your bones are connected and their direction aren't the same as in their transform matrix. It seems to me as if transform matrices are not changed when the parent-child bones are connected to each other. Thats what Im asking: can I safely do that and assume the matrices aren't changed?
If i understood your following sentence than it seems like it:
[quote]It's not stored in the world-coordinate bone matrix. You can calculate the local-to-parent bone matrix if you want.[/quote]
I'm not really good with the terminology. Is 'world-ccordinate bone matrix' the same as what I mean b 'absolute matrix' (relative to center point of the skeleton)? When you say "it's not stored in the world-coordinate bone matrix", can i safely assume that creating the bones with those matrices and connecting them after in my script won't modify the matrices?

[quote]Lauris71, i think, was trying to explain how you could create a process to convert all the bone matrices in your model and animation files to different matrices while not breaking the rig. It shouldn't be needed for what you are doing.[/quote]
Yes, it seems his doing the bone "connecting" manually by modifying the matrix and then changing it back before exporting.

Share this post


Link to post
Share on other sites
[quote name='joanp1' timestamp='1303319562' post='4800840']
can i safely assume that creating the bones with those matrices and connecting them after in my script won't modify the matrices?
[/quote]
Don't know unfortunately, as I've never dealt with blender scripts. Just try it. It doesn't take that long to just add that code to your script. If it does change the matrix, then instead of attaching using the world matrix, you'll have to calculate the local-to-parent matrix and use that to attach them ( inverse( parent_world ) * child_world ). But I have no idea how the attachments work in a blender script. sorry

The end result you are looking for should be all your bones attached, yet having the exact same orientation matrix as your original model file you imported without attaching them.

Share this post


Link to post
Share on other sites
[quote name='joanp1' timestamp='1303536341' post='4801847']
Yes! I just tested it and it's true. Thank you again. But Im afraid I still need to know how you handle the outmost bones?
[/quote]
There is nothing special about those, they are just attached to their parent as is.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this