Try this for general matrix/quaternion questions and answers:

[link]
http://skal.planet-d.net/demo/matrixfaq.htm
[/link]

You can find a very optimal way of performing a 4x4 inverse here:

[link]
http://developer.intel.com/design/pentiumiii/sml/24504301.pdf
[/link]

According to Intel, performing a matrix inverse yields faster results for Cramer''s rule than Gaussian elimination with dimensions <= 6. All the information and source code is available at the above link.

Bear in mind also that if your 4x4 matrices are *only* a combination of any of the following: translation, rotation, or scale, you can perform an "affine matrix" inverse much quicker than the above methods. Information available here:

[link]
http://www.cs.unc.edu/~gotz/code/affinverse.html
[/link]

HTH