Johnny Chung Lee (who uploaded this video) used the Wii mote for head tracking, I want to do the same thing with a Kinect & OpenNI.
However, my problem isn't related to the kinect or head tracking...
My problem is that I'm having a hard time to wrap my head around setting up correct View and Projection matrices to look into a tunnel as shown in the youtube video.
Here's an example:
Right now I just want to get this step to work, so I'm using keyboard input to move my camera around.
It doesn't make much sense to do any head tracking before I got this working.
My problem is figuring out the correct View & Projection matrices for an arbitrary camera position in front of the TV screen, so the camera actually looks "through the screen".
Here is what I tried:
//Position = Camera Position View = Matrix.CreateLookAt(Position, new Vector3(Position.X, Position.Y, 0), Vector3.Up); //Create a perspective off center //The tunnel model is 16f wide and 9f high (My TV is 16:9, resolution is 1920x1080) //near and far planes are 0.05f & 500f float left = nearPlane * (-16f + Position.X) / Position.Z; float right = nearPlane * (16f + Position.X) / Position.Z; float bottom = nearPlane * (-9f - Position.Y) / Position.Z; float top = nearPlane * (9f - Position.Y) / Position.Z; Projection = Matrix.CreatePerspectiveOffCenter(left, right, bottom, top, nearPlane, farPlane);
Oddly enough this seems to work on the Y axis, but not on the X axis?
If I move the camera up or down I get the wanted effect.
However if I move it left or right it just scrolls past the edge of the tunnel (the perspective does not change). :/
I think I'm missing something here, but I can't figure out what it is.
Any ideas / suggestions what I could try?