Oh okay, so the reason that the view matrix isn't what I think it is is because we are trying to perform the opposite rotations that apply to the camera and that happens to be the transpose of the 3x3 rotation matrix? So if the camera's view is rotated to the left by 90 the degrees, the view matrix will contain a rotation by -90 degrees instead right?
Yeah, the easist way that I find to create a view matrix is to construct a "local-to-world" (aka world) matrix as if the camera was an object in the world, and then invert this matrix to get a "world-to-camera" (aka view) matrix.
If a 3x3 matrix only contains the three axis, then transposing it is the same as inverting it (and cheaper).
One thing to always remember is that OpenGL/DirectX always expect element 12,13,14,15 of the matrix to be translation. However you decide to use the other elements is completely up to you, as long you remain consistent
No they don't.
You can use column-major maths, which looks on paper like:
$$\begin{bmatrix} Right.x & Up.x & Forward.x & Pos.x \\ Right.y & Up.y & Forward.y & Pos.y \\ Right.z & Up.z & Forward.z & Pos.z \\ 0 & 0 & 0 & 1 \end{bmatrix}$$
or row-major maths, which looks on paper like:
$$\begin{bmatrix} Right.x & Right.y & Right.z & 0 \\ Up.x & Up.y & Up.z & 0 \\ Forward.x & Forward.y & Forward.z & 0 \\ Pos.x & Pos.y & Pos.z & 1 \end{bmatrix}$$
And you can use column-major arrays, or row-major arrays.
If you use column-major maths with column-major arrays, or if you use row-major maths with row-major arrays, then your array of 16 floats will look like:
Right.x, Right.y, Right.z, 0, Up.x , Up.y, Up.z, 0, Forward.x, Forward.y, Forward.z, 0, Pos.x, Pos.y, Pos.z, 1
If you use column-major maths with row-major arrays, or if you use row-major maths with column-major arrays, then your array of 16 floats will look like:
Right.x, Up.x, Forward.x, Pos.x, Right.y, Up.y, Forward.y, Pos.y, Right.z, Up.z, Forward.z, Pos.z, 0, 0, 0, 1
All four of those choices of conventions are supported by D3D and OpenGL.
The mathematical convention alters how you write your math, e.g. whether you write vOut = vIn * projection * view * world, or vOut = world * view * projection * vIn.
The array convention alters how you write your matrix library, and whether you write column_major float4x4 myMatrix; or row_major float4x4 myMatrix; in your shader code.
If you're using an existing matrix library, then both of these choices may have already been made for you.