I'm not sure why you have a seperate "skinMatrix" and "transformMatrix" being applied to the animation:
Export your vertex data post-transformed, so that the model looks correct when skinned with identity matricies.
Export the inverse of the worldspace bind pose for each joint.
This matrix is used to bring the vertex back into local space as necessary.
At runtime, the skinning matrix is the animated worldspace matrix multiplied by the inverse-bind-pose matrix.
If the worldspace matrices match the original bind pose, you get identity matrices and so your model is correct. If you move the joints around, you will see the mesh move correctly.
The local-space animation matrix is built by composing a rotation matrix from the quaternion (which should just end up being the top left 3x3), multiplying the topleft 3x3 by the scale components (no need to multiply the bottom row or right column) then setting the translation row (or column depending on whether your matrices are transposed) You are better off writing the matrix manually instead of multiplying in the data:
On the note of inverse bind pose matrices, the big reason to stick with storing the inverse bind pose and leaving the mesh data in transformed space on export is for multiple influences per vertex. If you were to transform the mesh into local space, it would be a unique local space per bone influence.
When dealing with multiple influences you can either run the vertex through each of the matrices separately, then blend the results, or blend the matrices together then multiply the vertex through it. There are other ways of doing this blending that give nicer results, less 'candy wrapping' etc, and are worth looking at once you have your core steps up and running.