Sign in to follow this  
tmason

OpenGL Combining(?) Quaterions Accurately from Keyboard/Mouse and other sources ...

Recommended Posts

tmason    326

Hello,

 

I would like to combine mouse and keyboard inputs with the Oculus Rift to create a smooth experience for the user. The goals are:

  • Positional movement 100% controlled by the keyboard relative to the direction the person is facing.
  • Orientation controlled 100% by HMD devices like the Oculus Rift.
  • Mouse orbit capabilities adding to the orientation of the person using the Oculus Rift. For example, if I am looking left I can still move my mouse to "move" more leftward.

Now, I have 100% working code for when someone doesn't have an Oculus Rift, I just don't know how to combine the orientation and other elements of the Oculus Rift to my already working code to get it 100%.

 

Not good on the math.

 

Anyway, here is my working code for controlling the keyboard and mouse without the Oculus Rift:

 

Note that all of this code assumes a perspective mode of the camera:

/*

Variables

*/

glm::vec3 DirectionOfWhereCameraIsFacing;
glm::vec3 CenterOfWhatIsBeingLookedAt;
glm::vec3 PositionOfEyesOfPerson;
glm::vec3 CameraAxis;
glm::vec3 DirectionOfUpForPerson;
glm::quat CameraQuatPitch;
float     Pitch;
float	  Yaw;
float	  Roll;
float     MouseDampingRate;
float     PhysicalMovementDampingRate;
glm::quat CameraQuatYaw;
glm::quat CameraQuatRoll;
glm::quat CameraQuatBothPitchAndYaw;
glm::vec3 CameraPositionDelta;

/*

Inside display update function.

*/

DirectionOfWhereCameraIsFacing = glm::normalize(CenterOfWhatIsBeingLookedAt - PositionOfEyesOfPerson);
CameraAxis = glm::cross(DirectionOfWhereCameraIsFacing, DirectionOfUpForPerson);
CameraQuatPitch = glm::angleAxis(Pitch, CameraAxis);
CameraQuatYaw = glm::angleAxis(Yaw, DirectionOfUpForPerson);
CameraQuatRoll = glm::angleAxis(Roll, CameraAxis);
CameraQuatBothPitchAndYaw = glm::cross(CameraQuatPitch, CameraQuatYaw);
CameraQuatBothPitchAndYaw = glm::normalize(CameraQuatBothPitchAndYaw);
DirectionOfWhereCameraIsFacing = glm::rotate(CameraQuatBothPitchAndYaw, DirectionOfWhereCameraIsFacing);
PositionOfEyesOfPerson += CameraPositionDelta;
CenterOfWhatIsBeingLookedAt = PositionOfEyesOfPerson + DirectionOfWhereCameraIsFacing * 1.0f;
Yaw *= MouseDampingRate;
Pitch *= MouseDampingRate;
CameraPositionDelta = CameraPositionDelta * PhysicalMovementDampingRate;
View = glm::lookAt(PositionOfEyesOfPerson, CenterOfWhatIsBeingLookedAt, DirectionOfUpForPerson);
ProjectionViewMatrix = Projection * View;

The Oculus Rift provides orientation data via their SDK and can be accessed like so:

/*

Variables

*/

ovrMatrix4f OculusRiftProjection;
glm::mat4	Projection;
OVR::Quatf	OculusRiftOrientation;
glm::quat	CurrentOrientation;

/*

Partial Code for retrieving projection and orientation data from Oculus SDK

*/

OculusRiftProjection = ovrMatrix4f_Projection(MainEyeRenderDesc[l_Eye].Desc.Fov, 10.0f, 6000.0f, true);

for (int o = 0; o < 4; o++){
	for (int i = 0; i < 4; i++) {
		Projection[o][i] = OculusRiftProjection.M[o][i];
	}
}

Projection = glm::transpose(Projection);

OculusRiftOrientation = PredictedPose.Orientation.Conj();

CurrentOrientation.w = OculusRiftOrientation.w;
CurrentOrientation.x = OculusRiftOrientation.x;
CurrentOrientation.y = OculusRiftOrientation.y;
CurrentOrientation.z = OculusRiftOrientation.z;

CurrentOrientation = glm::normalize(CurrentOrientation);

After that last line the glm based quaterion "CurrentOrientation" has the correct information which, if plugged straight into the MVP matrix and sent into OpenGL will allow you to move your head around in the environment without issue.

 

Now, my problem is how to combine the two parts together successfully.

 

When I have done this in the past it results in the rotation stuck in place (when you turn your head left you keep rotating left as opposed to just rotating in the amount that you turned) and the fact that I can no longer accurately determine the direction the person is facing so that my position controls work.

 

How can I successfully achieve my goals?

 

Thank you for your time and all of your help thus far.

 

Very much appreciated.

Edited by tmason

Share this post


Link to post
Share on other sites
tmason    326

Hello to All,

So I completely resolved the problem down to a simple issue of getting the correct rotation.

I think a video explains it best: http://tinypic.com/player.php?v=nyi7g2%3E&s=8

The link takes you to a free video hosting service tinypic.com.

What happens is that rotating left or right is OK until after you go past approximately 75 degrees in which case the viewpoint goes crazy.

Additionally, looking upwards in the rift actually makes you look down and looking down makes you look up. It also rotates crazily after 75 degrees.

Any ideas? Thank you for your time.

Anyway, here is the new code:

/*
 
Variables
 
*/
 
glm::vec3 DirectionOfWhereCameraIsFacing;
glm::vec3 RiftDirectionOfWhereCameraIsFacing;
glm::vec3 CenterOfWhatIsBeingLookedAt;
glm::vec3 RiftCenterOfWhatIsBeingLookedAt;
glm::vec3 PositionOfEyesOfPerson;
glm::vec3 CameraAxis;
glm::vec3 DirectionOfUpForPerson;
glm::quat CameraQuatPitch;
float     Pitch;
float	  Yaw;
float	  Roll;
float     MouseDampingRate;
float     PhysicalMovementDampingRate;
glm::quat CameraQuatYaw;
glm::quat CameraQuatRoll;
glm::quat CameraQuatBothPitchAndYaw;
glm::vec3 CameraPositionDelta;
 
/*
 
Inside display update function.
 
*/
 
DirectionOfWhereCameraIsFacing = glm::normalize(CenterOfWhatIsBeingLookedAt - PositionOfEyesOfPerson);
CameraAxis = glm::cross(DirectionOfWhereCameraIsFacing, DirectionOfUpForPerson);
CameraQuatPitch = glm::angleAxis(CurrentCameraViewingSettings.Pitch, CameraAxis);
CameraQuatYaw = glm::angleAxis(CurrentCameraViewingSettings.Yaw, DirectionOfUpForPerson);
CameraQuatRoll = glm::angleAxis(CurrentCameraViewingSettings.Roll, CameraAxis);
CameraQuatBothPitchAndYaw = glm::cross(CameraQuatPitch, CameraQuatYaw);
CameraQuatBothPitchAndYaw = glm::normalize(CameraQuatBothPitchAndYaw);
DirectionOfWhereCameraIsFacing = glm::rotate(CameraQuatBothPitchAndYaw, DirectionOfWhereCameraIsFacing);
CameraUpdateMutex.lock();
RiftDirectionOfWhereCameraIsFacing = DirectionOfWhereCameraIsFacing;
RiftDirectionOfWhereCameraIsFacing = glm::rotate(CurrentOrientation, DirectionOfWhereCameraIsFacing);
CameraUpdateMutex.unlock();
PositionOfEyesOfPerson += CameraPositionDelta;
CenterOfWhatIsBeingLookedAt = PositionOfEyesOfPerson + DirectionOfWhereCameraIsFacing * 1.0f;
CameraUpdateMutex.lock();
RiftCenterOfWhatIsBeingLookedAt = PositionOfEyesOfPerson + RiftDirectionOfWhereCameraIsFacing * 1.0f;
CameraUpdateMutex.unlock();
CurrentCameraViewingSettings.Yaw *= CurrentCameraViewingSettings.MouseDampingRate;
CurrentCameraViewingSettings.Pitch *= CurrentCameraViewingSettings.MouseDampingRate;
CameraPositionDelta = CameraPositionDelta * CurrentCameraViewingSettings.PhysicalMovementDampingRate;
View = glm::lookAt(PositionOfEyesOfPerson, CenterOfWhatIsBeingLookedAt, DirectionOfUpForPerson);

And here is the updated code within the display function.

Again, partial listing:

/*
 
Variables
 
*/
 
ovrMatrix4f OculusRiftProjection;
glm::mat4	RiftProjection;
OVR::Quatf	OculusRiftOrientation;
glm::quat	CurrentOrientation;
glm::vec3	CurrentEulerAngles;
glm::mat4	RiftView;
glm::mat4	RiftProjectionViewMatrix;
 
/*
 
Partial Code for retrieving projection and orientation data from Oculus SDK
 
*/
 
ovrEyeType l_Eye = (*MainRiftDeviceDesc).EyeRenderOrder[l_EyeIndex];
ovrPosef l_EyePose = ovrHmd_BeginEyeRender((*MainRiftDevice), l_Eye);
OVR::Posef PredictedPose = ovrHmd_GetSensorState((*MainRiftDevice), m_HmdFrameTiming.ScanoutMidpointSeconds).Predicted.Pose;
 
glViewport(MainEyeRenderDesc[l_Eye].Desc.RenderViewport.Pos.x,
	MainEyeRenderDesc[l_Eye].Desc.RenderViewport.Pos.y,
	MainEyeRenderDesc[l_Eye].Desc.RenderViewport.Size.w,
	MainEyeRenderDesc[l_Eye].Desc.RenderViewport.Size.h);
 
OculusRiftProjection = ovrMatrix4f_Projection(MainEyeRenderDesc[l_Eye].Desc.Fov, 10.0f, 6000.0f, true);
 
for (int o = 0; o < 4; o++){
	for (int i = 0; i < 4; i++) {
		RiftProjection[o][i] = OculusRiftProjection.M[o][i];
	}
}
 
RiftProjection = glm::transpose(RiftProjection);
 
OculusRiftOrientation = PredictedPose.Orientation;
 
OculusRiftOrientation.GetEulerAngles<OVR::Axis_X, OVR::Axis_Y, OVR::Axis_Z, OVR::Rotate_CCW, OVR::Handed_R>
	(&CurrentEulerAngles.x, &CurrentEulerAngles.y, &CurrentEulerAngles.z);
 
CurrentOrientation = glm::quat(CurrentEulerAngles);
 
RiftView = glm::lookAt(PositionOfEyesOfPerson, RiftCenterOfWhatIsBeingLookedAt, DirectionOfUpForPerson);
RiftView = glm::translate(RiftView, glm::vec3(l_EyePose.Position.x, l_EyePose.Position.y, l_EyePose.Position.z));
RiftProjectionViewMatrix = RiftProjection * RiftView;

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this  

  • Similar Content

    • By Zaphyk
      I am developing my engine using the OpenGL 3.3 compatibility profile. It runs as expected on my NVIDIA card and on my Intel Card however when I tried it on an AMD setup it ran 3 times worse than on the other setups. Could this be a AMD driver thing or is this probably a problem with my OGL code? Could a different code standard create such bad performance?
    • By Kjell Andersson
      I'm trying to get some legacy OpenGL code to run with a shader pipeline,
      The legacy code uses glVertexPointer(), glColorPointer(), glNormalPointer() and glTexCoordPointer() to supply the vertex information.
      I know that it should be using setVertexAttribPointer() etc to clearly define the layout but that is not an option right now since the legacy code can't be modified to that extent.
      I've got a version 330 vertex shader to somewhat work:
      #version 330 uniform mat4 osg_ModelViewProjectionMatrix; uniform mat4 osg_ModelViewMatrix; layout(location = 0) in vec4 Vertex; layout(location = 2) in vec4 Normal; // Velocity layout(location = 3) in vec3 TexCoord; // TODO: is this the right layout location? out VertexData { vec4 color; vec3 velocity; float size; } VertexOut; void main(void) { vec4 p0 = Vertex; vec4 p1 = Vertex + vec4(Normal.x, Normal.y, Normal.z, 0.0f); vec3 velocity = (osg_ModelViewProjectionMatrix * p1 - osg_ModelViewProjectionMatrix * p0).xyz; VertexOut.velocity = velocity; VertexOut.size = TexCoord.y; gl_Position = osg_ModelViewMatrix * Vertex; } What works is the Vertex and Normal information that the legacy C++ OpenGL code seem to provide in layout location 0 and 2. This is fine.
      What I'm not getting to work is the TexCoord information that is supplied by a glTexCoordPointer() call in C++.
      Question:
      What layout location is the old standard pipeline using for glTexCoordPointer()? Or is this undefined?
       
      Side note: I'm trying to get an OpenSceneGraph 3.4.0 particle system to use custom vertex, geometry and fragment shaders for rendering the particles.
    • By markshaw001
      Hi i am new to this forum  i wanted to ask for help from all of you i want to generate real time terrain using a 32 bit heightmap i am good at c++ and have started learning Opengl as i am very interested in making landscapes in opengl i have looked around the internet for help about this topic but i am not getting the hang of the concepts and what they are doing can some here suggests me some good resources for making terrain engine please for example like tutorials,books etc so that i can understand the whole concept of terrain generation.
       
    • By KarimIO
      Hey guys. I'm trying to get my application to work on my Nvidia GTX 970 desktop. It currently works on my Intel HD 3000 laptop, but on the desktop, every bind textures specifically from framebuffers, I get half a second of lag. This is done 4 times as I have three RGBA textures and one depth 32F buffer. I tried to use debugging software for the first time - RenderDoc only shows SwapBuffers() and no OGL calls, while Nvidia Nsight crashes upon execution, so neither are helpful. Without binding it runs regularly. This does not happen with non-framebuffer binds.
      GLFramebuffer::GLFramebuffer(FramebufferCreateInfo createInfo) { glGenFramebuffers(1, &fbo); glBindFramebuffer(GL_FRAMEBUFFER, fbo); textures = new GLuint[createInfo.numColorTargets]; glGenTextures(createInfo.numColorTargets, textures); GLenum *DrawBuffers = new GLenum[createInfo.numColorTargets]; for (uint32_t i = 0; i < createInfo.numColorTargets; i++) { glBindTexture(GL_TEXTURE_2D, textures[i]); GLint internalFormat; GLenum format; TranslateFormats(createInfo.colorFormats[i], format, internalFormat); // returns GL_RGBA and GL_RGBA glTexImage2D(GL_TEXTURE_2D, 0, internalFormat, createInfo.width, createInfo.height, 0, format, GL_FLOAT, 0); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); DrawBuffers[i] = GL_COLOR_ATTACHMENT0 + i; glBindTexture(GL_TEXTURE_2D, 0); glFramebufferTexture(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0 + i, textures[i], 0); } if (createInfo.depthFormat != FORMAT_DEPTH_NONE) { GLenum depthFormat; switch (createInfo.depthFormat) { case FORMAT_DEPTH_16: depthFormat = GL_DEPTH_COMPONENT16; break; case FORMAT_DEPTH_24: depthFormat = GL_DEPTH_COMPONENT24; break; case FORMAT_DEPTH_32: depthFormat = GL_DEPTH_COMPONENT32; break; case FORMAT_DEPTH_24_STENCIL_8: depthFormat = GL_DEPTH24_STENCIL8; break; case FORMAT_DEPTH_32_STENCIL_8: depthFormat = GL_DEPTH32F_STENCIL8; break; } glGenTextures(1, &depthrenderbuffer); glBindTexture(GL_TEXTURE_2D, depthrenderbuffer); glTexImage2D(GL_TEXTURE_2D, 0, depthFormat, createInfo.width, createInfo.height, 0, GL_DEPTH_COMPONENT, GL_FLOAT, 0); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); glBindTexture(GL_TEXTURE_2D, 0); glFramebufferTexture(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, depthrenderbuffer, 0); } if (createInfo.numColorTargets > 0) glDrawBuffers(createInfo.numColorTargets, DrawBuffers); else glDrawBuffer(GL_NONE); if (glCheckFramebufferStatus(GL_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE) std::cout << "Framebuffer Incomplete\n"; glBindFramebuffer(GL_FRAMEBUFFER, 0); width = createInfo.width; height = createInfo.height; } // ... // FBO Creation FramebufferCreateInfo gbufferCI; gbufferCI.colorFormats = gbufferCFs.data(); gbufferCI.depthFormat = FORMAT_DEPTH_32; gbufferCI.numColorTargets = gbufferCFs.size(); gbufferCI.width = engine.settings.resolutionX; gbufferCI.height = engine.settings.resolutionY; gbufferCI.renderPass = nullptr; gbuffer = graphicsWrapper->CreateFramebuffer(gbufferCI); // Bind glBindFramebuffer(GL_DRAW_FRAMEBUFFER, fbo); // Draw here... // Bind to textures glActiveTexture(GL_TEXTURE0); glBindTexture(GL_TEXTURE_2D, textures[0]); glActiveTexture(GL_TEXTURE1); glBindTexture(GL_TEXTURE_2D, textures[1]); glActiveTexture(GL_TEXTURE2); glBindTexture(GL_TEXTURE_2D, textures[2]); glActiveTexture(GL_TEXTURE3); glBindTexture(GL_TEXTURE_2D, depthrenderbuffer); Here is an extract of my code. I can't think of anything else to include. I've really been butting my head into a wall trying to think of a reason but I can think of none and all my research yields nothing. Thanks in advance!
    • By Adrianensis
      Hi everyone, I've shared my 2D Game Engine source code. It's the result of 4 years working on it (and I still continue improving features ) and I want to share with the community. You can see some videos on youtube and some demo gifs on my twitter account.
      This Engine has been developed as End-of-Degree Project and it is coded in Javascript, WebGL and GLSL. The engine is written from scratch.
      This is not a professional engine but it's for learning purposes, so anyone can review the code an learn basis about graphics, physics or game engine architecture. Source code on this GitHub repository.
      I'm available for a good conversation about Game Engine / Graphics Programming
  • Popular Now