Gifaraj, I think you're trying to solve something that -even if it *is* soluble- it doesn't need to be solved!
You say that multiplying the top order matrix by the scale matrix would fix the problem, just like you do when you export things on your own. This multiplication however, doesn't alter the product of the matrices by affecting each single one of them.
What is the point in looking for changes in each matrix, when only 4 numbers in the right-most one have changed sign? The others don't change!
I believe that trying to figure out how each matrix is affected by this negation, and counteract it, is meaningless. The other matrices are not affected directly. They will only yield a different product. Trying to find some formula to predict the effects of this on the other matrices is utopic.
The problem you present is pretty much the same as having a sequence of integers that are to be multiplied with each other, and trying to find out, how negating one of them would affect all others, in order to be able to compensate for it when calculating their product, without negating the original number.
Only one number is being affected, nothing happened to the others. It will only affect their product. What is to compensate for in the other numbers, when only one of them has changed?
I hope you see what I mean. The actual problem is that there
is no problem. How you will deal with it in software, I don't know. Can't you just negate the top level matrix of each frame-hierarchy upon import?
[edit:
You want to be able to calculate the effects of the change of system's handedness in each matrix in a hierarchy, when there aren't any. Only the first one needs adjustment.]
Quote:
Look at http://www.okino.com/conv/exp_xof.htm the section where it says "Method to Convert from Right-Handed Coordinate System to Left-Handed Coordinates". The last method, "Flip Z of Top-Level Matrix", is what I have been doing before. I want to do what the other method, "Flip Z of Vectors and Reverse Angle of Quaternions", does.
Negating the angle of some rotations is necessary when porting data from CSs of different handedness. E.g. if you have negated the Z axis, all previous Z rotations must now act on the negated Z axis. This is achieved easier, by simply negating the angle of rotation. If you had swapped axes too, you should also adjust them to act on the new axes.
In your case, implicit rotations that are hardcoded in matrices are handled through negation of Z, just like the "Flip Z of top-level". Explicit rotations, like "Quaternion(cos(π/4), sin(π/4)*{0,0,1})" need either the axis or the angle to be negated. In any case, you replace all quaternions with their conjugate.
I don't think that this application does anything more than what it says it does.