I've spent the day trying to implement a system in an SDL2 OpenGL app that rotates the actual camera by pitch, yaw and roll.

I'm fed up of reading tutorials that use glm::lookat and set the up vector to (0, 1, 0), and forgo the ability to roll the camera, trying to emulate a first person camera.

Let me first define what I'm trying to do. I noticed naively that using a glm::rotate matrix as the view matrix just rotates the objects in the scene, and not the camera. I actually want the camera to rotate, so if pitch is 90 degrees, the camera looks down. Or if yaw is set to 90 degrees, the camera looks right, and therefore cannot see the objects. If roll is set to 90 degrees the scene is viewed as rotated sideways.

Considering that I'm not afraid of quaternions and I know they remove gimbal lock if used correctly, what is the best way to create a view matrix that rotates the camera itself by specifying pitch, yaw and roll?

Thank you for any help!

What part are you hung up on? what are you trying? There are a few ways to get a view matrix depending on your setup.

If you treat a camera as a regular object like any other, the view matrix is simply the inverse of that objects world matrix. Controlling it with pitch yaw roll (p-y-r), first thing youll want to do is track these values and wrap them at 360.

Once you have p-y-r angles, make rotations out of them (using quaternions, matrices) around each axis from the axis|angle, roll around z, yaw around y etc. Concatenate them, world = roll * (yaw * pitch), invert and convert to matrix if not already, and load it as the view matrix.

Other options, track the y-p-r values, use the angles to rotate the x-y-z axis, use the resulting vectors to create the vectors for the lookat function, at = pos + zaxis, up = yaxis. You could also use the final matrix from the first option to rotate the x-y-z axis, and build the lookat vectors from the results.

So I did the thing with an inverse rotation matrix. But it just rotates the objects in the scene inversely. As opposed to the camera. That is if I set the pitch to 90 degrees, I can still see the objects, just rotated. I should be looking straight down, and therefore nothing, since my scene is just a couple of cubes.

glm::mat4 cam = glm::rotate(glm::mat4(1.0), renderingData.CameraRotation.x, glm::vec3(1, 0, 0));
cam = glm::rotate(cam, renderingData.CameraRotation.y, glm::vec3(0, 1, 0));
cam = glm::rotate(cam, renderingData.CameraRotation.z, glm::vec3(0, 0, 1));
    
glm::mat4 camMove = glm::translate(glm::mat4(1.0), renderingData.CameraPosition);
    
glm::mat4 mvp = glm::perspective(30.0f, 800.0f / 450.0f, 0.20f, 1000.0f) * camMove * glm::inverse(cam) * model;

Sorry currently I have no time ti explain this is a simple code that might help:

struct camera
{
    float radius = 1.f;
    float yaw = 0, pitch = 0;

    void update(vec3f& pos, vec3f& up)
    {
        pos = vec3f(cos(yaw) * cos(pitch), sin(pitch), -sin(yaw) * cos(pitch)) * radius;

        up =
        normalized(
            vec3f(cos(yaw) * cos(pitch + half_pi<float>()),
            sin(pitch + half_pi<float>()),
            -sin(yaw) * cos(pitch + half_pi<float>()))
        );
    }

} g_camera;

Basically to compute the up and look at I trasform those yall and pitch from spherical coordinate system to R^3


Edit : the camera is orbiting around 0 so the look would be normalized(-pos)

Try incorporating the camera translation matrix into the cam matrix before the invert, you may be moving the object down after its in view space.

You really need to read more about how the World, View and Projection matrices are used in 3D development, especially how they would be used for a first person camera, if that is what you're after. And reading code doesn't count.

It sounds to me like you only read and implemented a couple of beginner tutorials which usually teach you only how to put stuff on the screen, in which case a simple view matrix which makes the camera point forward is sufficient.

The rotation and translation mechanics of a first person camera, which would be a specific implementation of the view matrix are not usually covered by such tutorials.

Don't constrain yourself to just reading tutorials from the GLM documentation. The theory behind those matrices is not only specific to GLM. Any 3D graphics library or API uses the same concept of World, View and Projection matrices. And most likely, none of those helper libraries like GLM go into detail about the theory. They just throw you into the code assuming you already know how matrices work. Although it sounds to me more like you did not even try at all to understand how the matrices are used to implement a first person camera, rather than that you read code that does what you want. You can't just wing it and hope it works.

Considering that I'm not afraid of quaternions and I know they remove gimbal lock if used correctly, what is the best way to create a view matrix that rotates the camera itself by specifying pitch, yaw and roll?

To give you a nudge in the right direction: the problem is not HOW you create the matrices, but WHEN you use them. You already have the HOW part covered by using the glm::rotate() function. And from the theory, it should already be obvious that if your "cam" matrix affects the objects instead of the final scene, it's because you're concatenating (multiplying) your "cam" matrix too early on as part of the final, composite matrix.

Thank you all for your input, this is most fascinating. I had some success with combining the rotation and translation matrices together when getting the inverse. However, I still want to learn more about the real maths though, so I'm going to try implementing my own simple maths library to experiment with. And I'll get watching those Khan academy videos.

Thank you!