But take a look at how OpenGL's matrices are commonly designed here in appendix F. It lists their inverse also.

I tried implementing my own fucntions to generate a matrix based on those equations, as well as an inverse:

Multiplying a matrix together with it's own inverse should result in an identity matrix. I'm getting mostly identity, except col 3 row 4 is always -5.9, and col 4 row 3 is always -1.2. I've tripple-checked that my matrices match the ones in the image.

I think the problem is in my matrix multiplication. I know how to multiply the 3x3 rotation section of the matrix and then add the translation, but what do I do with the perspective section?

Edit: I've been googling and even looked into a few CG programming books. nobody seems to say how to handle the bottom row of a GL 4x4 when multiplying two matrices together...