Although I haven't noticed so far, I think there are errors in my LookAt function. AFAIK the resulting matrix should be orthonormal and the scaling effect after transposition suggests it is not. I must look into it. However I would be very happy if You could help me. Here is the code:

Please bear in mind that some aspects of the implementation are not clear yet. So I must guess some things...

public static TransformationMatrix calculateLookAtMatrix(Vector3d eye, Vector3d center, Vector3d up)
{
float [] matrix = new float[16];
Vector3d forward = Vector3d.sub(center , eye);
forward.normalize();

I assume that Vector3d.sub(a, b) computes a-b (in this order).

Vector3d side = Vector3d.cross(forward , up);
side.normalize();
Vector3d plane_up = Vector3d.cross(side , forward); plane_up.normalize();

Normalizing "plane_up" is not strictly necessary, because both arguments "side" and "forward" have unit length and "side" is guaranteed to be orthogonal to "forward".

matrix[0] = side.getX();
matrix[1] = plane_up.getX();
matrix[2] = -forward.getX();
matrix[3] = 0.0f;
matrix[4] = side.getY();
matrix[5] = plane_up.getY();
matrix[6] = -forward.getY();
matrix[7] = 0.0f;
matrix[8] = side.getZ();
matrix[9] = plane_up.getZ();
matrix[10] = -forward.getZ();
matrix[11] = 0.0f;
matrix[12] = 0.0f;
matrix[13] = 0.0f;
matrix[14] = 0.0f;
matrix[15] = 1.0f;

Now things become a bit complicated.

1. You're negating (solely) the forward direction, so your camera is defined to look along its local negative z direction, right?

2. Because you're about to compute the transposed matrix, from the indexing I conclude that you're using the column major layout, right? I mean that the matrix indices are arranged like so:

[ 0 4 8 12 ]

[ 1 5 9 13 ]

[ 2 6 10 14]

[ 3 7 11 15]

If that is right it would further mean that you are using column vectors.

Vector3d neweye = new Vector3d(eye);
neweye.reverse();
TransformationMatrix ret = new TransformationMatrix(matrix);
ret.multiply(TransformationMatrix.calculateTranslationMatrix(neweye));

The ret.multiply computes probably

**R**^{t} *

**T**( -eye ) here. This order would be correct if and only if you're using column vectors. Do you do so? Unfortunately, nothing in the shown code gives a clear criterion whether you do so.

return ret;
}

Nothing to say here