Inverse( Inverse(T) ) should return T. The scalar and quaternion components are correct, however the translation vector is off. It looks like the method to find the inverse translation component might be incorrect.
Thanks guys. After much experimentation, the problem was with rounding errors. I have since switch to using a fixed point number representation for greater accuracy and have solved the issue, pretty much as AB suggested.
Also, Krypt0n you migh be right about the error introduced by incrementing xs and xe each iteration, I'll check it out.