My theory is that it's a big cost to compute/transfert/stock all the inverse matrices, so it's better to have a normal slightly false rather than an exact normal but a lower FPS.
Is my theory true? or am I missing something?

Almost, but not quite. If there is no scaling in the matrix, the inverse transpose, is the same as the original matrix. Scaling also introduces a number of additional complexities when computing the world space bone matrices (correcting for negative scale etc). This tends to make the code a lot more complex, which makes everything run just a bit slower.... Most people therefore disallow scaling in their animation systems as a result.

One last thing about orthogonal matrices that I discovered during my research (maybe that will help someone... one day...):
In some articles I read that a matrix that contains a uniform scaling is orthogonal (and when we transform a vector, there is no need to inverse&transpose). This is false: the uniform scaling matrix isn't orthogonal!

A uniform scaling matrix is orthogonal, but it's not orthonormal though. If the matrix is orthonormal, the inverse-transpose is the same as the original. That is not the case for orthogonal matrices.

BUT it's almost true: actually, if and only if, we don't care the norm of the vector because we normalize the vector just after the transformation (so in 99% of cases):
in this case, even if the matrix contains a uniform scaling (so isn't orthogonal), we don't need to inverse&transpose.
This isn't true for non-uniform scaling matrices.

Almost true != true.

You've replaced the inverse transpose with a vector normalisation, which works in this case. You are still multiplying the vector by the inverse transpose, you're just doing it in a slightly different way....

The upper left 3x3 section of a uniform scaling matrix should always be orthogonal
However, that doesnt mean that the columns are unit length. the only such matrix is the identity matrix
1 0 0
0 1 0
0 0 1
all columns are unit length and orthogonal to each other

In the field of mathematics, the formal definitions are as follows:
orthogonal matrix: A matrix which has orthonormal column (and hence) row vectors. It is a property of orthogonal matrices that their inverse equals the transpose.
orthonormal matrix: There is no such thing.

The reason for the above definition is perhaps historical, and perhaps due to that matrices with orthogonal (but not necessarily normalized) column/row vectors do not have nice properties to be worth much for mathematicians in linear algebra, and the above definition stuck.

The 3D games programmers definition:
orthogonal matrix: A matrix which has orthogonal column (if you are a Matrix*vector guy) or row (if you are a vector*Matrix guy) vectors, but not necessarily normalized.
orthonormal matrix: A matrix which has orthonormal column (and hence row) vectors.

Note that if a matrix has orthogonal (but not normalized) column vectors, it does not follow that its row vectors would also be normalized (and vice versa).

One thing that both mathematicians and 3D programmers do agree are the definitions of a set of orthogonal vectors (a collection of vectors which are all perpendicular to each other), and a set of orthonormal vectors (a collection of vectors which are all normalized and perpendicular to each other).

The above ill-logical definitions are discussed in both books Geometric Tools for Computer Graphics and Mathematics for 3D Game Programming and Computer Graphics. As a programmer, I definitely prefer the programmers' definitions, and that is what I use in my MathGeoLib library (find the link in my signature).