I am having real problems with my rotations.

Here's my class that is responsible for a world object's rotation, position and directional vectors:

[source lang="cpp"]#ifndef OBJECT_H
#define OBJECT_H

//object.h
//Vector3 is just your standard Vector3 class

class Object {
public:
virtual void update(double seconds) = 0;

inline void setPosition(const Vector3& pos) { position_ = pos; }
inline void setPosition(float x, float y, float z) {
position_.x = x;
position_.y = y;
position_.z = z;
}
inline void setRotation(const Vector3& rot) { rotation_ = rot; calculateDirections(); }

inline Vector3 getPosition() const { return position_; }
inline Vector3 getRotation() const { return rotation_; }
inline Vector3 getForward() const { return forward_; }
inline Vector3 getRight() const { return right_; }

protected:
Vector3 position_, rotation_;
Vector3 forward_, right_;

void calculateDirections();
};

#endif // OBJECT_H

//object.cpp
#include "object.h"

void Object::calculateDirections() {
float pitch = degToRad(rotation_.x);
float yaw = degToRad(rotation_.y);
float roll = degToRad(rotation_.z);

forward_.x = -sin(yaw) * cos(pitch);
forward_.y = sin(pitch);
forward_.z = -cos(yaw) * cos(pitch);
normalize(forward_);

right_.x = cos(yaw) * cos(roll);
right_.y = sin(roll);
right_.z = -sin(yaw) * cos(pitch);
normalize(right_);
}[/source]

The first problem is, my forward vector points in the wrong direction, the completely opposite direction. I.e, it points towards the camera instead of away. I could just invert it but is that the proper solution or a "it just works" solution?

Here's my camera class:
[source lang="cpp"]#ifndef CAMERA_H
#define CAMERA_H

#include <GL/gl.h>

#include "mouse.h"
#include "maths/vector3.h"
#include "maths/maths_helper.h"
#include "object.h"

using namespace Maths;

class Camera : public Object {
public:
Camera();

void update(double seconds);
void transform();
void setTarget(const Vector3& target);

inline int getZoom() const { return zoom_; }
private:
Mouse* mouse_;
int zoom_;
float rotationSpeed_;
Vector3 target_;
};

#endif // CAMERA_H

//camera.cpp

#include "camera.h"

Camera::Camera()
: mouse_(Mouse::getMouse()), zoom_(2), rotationSpeed_(0.15f) {
calculateDirections();
}

void Camera::update(double seconds) {
if(mouse_->right || mouse_->left) {
rotation_.x += rotationSpeed_ * (float)mouse_->yMotion;
rotation_.y += rotationSpeed_ * (float)mouse_->xMotion;

clamp(rotation_.x, 10.0f, 70.0f);

calculateDirections();
}

zoom_ = -mouse_->scroll;
clamp(zoom_, 2, 20);
}

void Camera::transform() {
gluLookAt(position_.x, position_.y, position_.z,
target_.x, target_.y, target_.z,
0.0f, 1.0f, 0.0f);
}

void Camera::setTarget(const Vector3& target) {
target_ = target;
position_ = target_ - (forward_ * (float)zoom_);
}
[/source]

I set the Camera's target to my player's position. I expect the camera to be behind the player but instead it's in front of the player and stuck below the plane Y=0. I can fix it by inverting the forward vector and the pitch but again is that the proper solution or just a "it works" fix?

Not sure if there is a right or wrong way. For instance, in my code my orientation is represented by a single quaternion, from this I build my forward, up, right vectors. My code expects the negative Z to be forward, so I negate the forward.
Why dont you like the idea of negating the vector?

Not sure if there is a right or wrong way. For instance, in my code my orientation is represented by a single quaternion, from this I build my forward, up, right vectors. My code expects the negative Z to be forward, so I negate the forward.
Why dont you like the idea of negating the vector?

I'm not against the idea of negating it, I just didn't know why it fixed my problem and so didn't know whether it was the correct solution.

Hi Rarge,

It appears you're trying to use your Object's calculateDirections method to calculate the directions of your camera. This doesn't actually work.

The purpose of an object's transformation matrix (and thus Right/Forward vectors), is to figure out its orientation in the world, relative to the root of the world. Given that, your calculateDirections method appears correct.

The purpose of a camera's transformation matrix is not to locate the camera in the world, but to transform other objects into camera space. To do that, you actually need the inverse of the camera's matrix. Think about it this way, if I move my camera to the right, relative to the camera, everything else moved left. Similarly if I move my camera in the -Z direction to get closer to my objects, it's as if they moved in the +Z direction to get closer to my camera.

You're using the gluLookAt method to actually set the View Matrix, which calculates the view matrix for you correctly, however the fact that SetTarget uses the forward vector which is going to be computed backward is a likely area of concern.

That makes sense, so where should I put the responsibility of inversing everything? Should Object automatically inverse everything or should I change the way my camera works?

Maybe I misunderstood your original post then.

if I expand everything out the order is:

[source lang="cpp"]glMatrixMode(GL_MODELVIEW); //this is set at the start and never changed
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); //then the below is called in this order every frame
glLoadIdentity();

camera_.update(seconds);
camera_.transform();

player_.update(seconds);

player_.draw();

glfwSwapBuffers();[/source]

player_ is just a Player object which is just derived from the Object class posted earlier with a draw method which draws a cube.

I must be doing something wrong somewhere else (or I'm misunderstanding you) because the camera code above doesn't work as expected. The camera is viewing the player from below (i.e. the pitch needs inverting)

I think maybe I'm transforming the player wrong?
Here's the player's transformation:
[source lang="cpp"]glPushMatrix();
glTranslatef(position_.x, position_.y, position_.z);
glRotatef(rotation_.y, 0.0f, 1.0f, 0.0f);

//draw a cube

glPopMatrix();[/source]

I'm sorry if I'm being a bit ignorant ):

EDIT:
Aha, I realise the problem and understand it now. It was all because I forgot -10 and -70 degrees is the same as 350 degrees and 290 degrees. If I change the pitch limits to 290 - 350 degrees then everything is fine.

As the pitch is the rotation around the x axis, to get the camera in the position I wanted I must rotate it 290 degrees clockwise. I was trying to get everything to rotate 70 counter clockwise which is that same as -70 degrees clockwise.

Such a silly mistake to make, sorry for being a time waste.