Applications work correctly only when both the code and the data are correct. Have you looked at the data at various places and times while the program is running to verify it's correct? If the data is correct, does the code do what you expect it to do?

Note: Hopefully you're starting out with a very simple model - one or two bones, and the simplest of animations (no animation at all - no rotations, etc.) If you're doing initial testing of 100s of lines of code with anything more complicated than that, go back and do that. That will give you the benefit of knowing what the matrices should look like at run-time, and you can verify the data and the code quickly.

XMMATRIX offset = XMMatrixTranspose(XMLoadFloat4x4(&AssimpToDirectXMatrix(m_Aib->mOffsetMatrix))); // Transpose to get the correct col/row - opengl sucks

Can you explain what this code is intended to do, and why? Only you know what AssimpToDirectXMatrix does, what m_Aib is, and why you transpose a matrix that (by the function description) already appears to be compatible with your DirectX app.

It's also not clear why you do that every frame. Offset matrices don't change. It can be a Bone member variable initialized when you load the mesh hierarchy.

EDIT:

Am I doing the matrix math correctly?

The form of most** of the math seems to be correct. It all depends, of course, on the order you do the calculations, whether your hierarchy is correct, etc. You should've verified the math by testing the simple model case mentioned above.

** See the above question WRT the offset matrix.

You apparently ignored the first and most important paragraph.

Applications work correctly only when both the code and the data are correct. Have you looked at the data at various places and times while the program is running to verify it's correct? If the data is correct, does the code do what you expect it to do?

Simply put: Code that doesn't work cannot be debugged if you can't verify that the data you want to render is what your program is coded to render. Hope that's a bit clearer.

The questions asked above are particularly important because you transpose some matrices (apparently not others?) and "flip" axes (whatever that may mean). I'm guessing you haven't verified that you import data correctly, or that you've verified that the manipulations you do to the data once it's imported is needed and/or done correctly.

At this point, your post boils down to: "Here's some code. It's part of what I use to render some data. It doesn't work. What's wrong?" I.e., something like "I have a formula for a variable C that takes variables A and B as input, but C isn't equal to 4. Is my formula correct?"

If you're serious about getting code and data that works together correctly, I suggest you learn some debugging techniques to determine first if the data is imported as you think it is, whether you need to be manipulating the data as you do, whether you do it correctly, etc.

Note: You really should be using a simple model (a couple of boxes, each skinned to one of two bones, is much simpler than a cylinder), with a static animation (animation frames present but no rotation).

I have completely no idea what the transformation matrices should look like.

That's frequently a problem which is aided by a simple model with a simple animation. See below.

You can also add some debugging code to calculate a matrix with known parameters. For instance, I suspect you "flip" axes to convert from right-hand to left-hand system. If it applies in your case, a common conversion from a right-hand system to a left-hand system is: scale the z-axis by -1, and rotate about the x-axis by pi/4. You can calculate that matrix in your code and see what it looks like. E.g., mat = scalingMatrix( 1, 1, -1 ) * rotatationMatrix( axis( 1, 0, 0 ), pi/4 ). By calculating example values in your code, you have something to compare to (for instance) to see if transposition is needed and/or successful.

There are a few of them with complicated floating point numbers, which combined together in some black-magical way SHOULD give an Identity matrix in the end (at 0 keyframe at least).

Exactly! The model should render in a static (base) pose. If it doesn't, start looking at the values of the vertices. Vertices forming a box should be easily recognizable. 3 indexed vertices should form a triangle with the proper winding order. Etc. You should be able to do that in a few minutes, just by inspection - maybe pencil and paper.

If the model renders in a static pose, add a rotation animation of just a few degrees around a known axis. I.e., the box vertices should still be very close to their original positions after transformations are applied. If not, step through the process and find out where the values of the vertices go bad. Etc.

Assimp isn't reading the orientation good

Assimp is probably okay. When a problem with established software like assimp occurs, start by assuming that your code is incorrect and/or that you don't understand the data. You'll fix problems much more quickly that way. As mentioned, use a simple model and examine the data you're importing. Understand both what you want and what you're getting.

If you still have doubts, download the assimp viewer and see if it loads/displays the dae file correctly.

Now there are two other, seemingly much simpler bugs:
- There is no inter-bone translation
- General object orientation is wrong

In a very general sense, no translation could be a transposition problem. I.e., your code expects translation in the last row of a matrix, but the translation is, in fact, in the last column. That would also certainly cause something "general" to be "wrong."

As mentioned, take the time to understand the data you're importing, and what you expect for your code to work correctly. Consider, you've been dealing with some related problem or other for at least 4 days now - perhaps by hacking at the code to see if maybe you can stumble onto something that works. If, instead, you spend a couple hours to debug your app in a structured way, you'll save a whole boatload of development time.

If that interests you (and please say if it doesn't), take a look at your imported data. Are matrices transposed from what you expect? Does the data appear to be for a right-handed system and you want to use left-handed?

Good for you for keeping at it. Skeletal animation/skinned meshes are, at best, a difficult implementation.

My current approach is double-checking every single step the data goes through, assuming the data is correct. At this moment I'm quite scared of number crunching transformation matrices, this will be the last resort ;)

Do you, in fact, know what correct data looks like? Or, are you really checking that the code works with what you want (or hope) the data will look like? Just sayin', code is generated to handle data - not the other way 'round. When imported, the data will be what it is.

I understand that transformations can be intimidating. I spent a lot of time examining and even printing out data when I coded my first custom animation program. However, if you use a simple model with basic transforms, you may be pleasantly surprised how familiar you'll get with transformations as you look at them.

Keep at it!