Tracking with Kinect and Oculus Rift character in-game
Wouldn't surprise me if this isn't already done. I have seen a demo that used something to track a person but I don't recall what that something was, it might have been Kinect. If you are interested in full room movement you should check out the HTC/Valve Vive.
This is also very cool:
They have all sorts of things going on in this and various effects to make people feel like they are there. Unfortunately I can't find the link now but I've also seen some university doing full moving VR using a motion capture type setup but that really is a bit beyond your average home solution. Still, it does show what's possible.
My specific research was trying to mark real world objects with tags (not my video, but that is pretty much how mine looked) so the computer could track them. That way when you pick up a block in the real world you can feel the weight and shape of the block and the computer could replace the block with anything in the VR world. I used this to make a chess game where you move physical pieces to play against a virtual opponent. The problem I ran into was knowing where your hand were in the virtual world. You mostly had to feel around blindly to play. That is why I tried using leap motion, but it would completely fail to track the hands once you picked anything up or even when your hands were near the chess pieces.
I haven't used knect but I suspect trying to use it for VR would result in either jumpy simulation or, using some sort of smoothing, would result in a laggy vr experience. Tracking a tag can be done much more accurately and much faster but requires more setup.
The second and third oculus that is coming out uses infrared lights on the headset as markers and a camera tracks the positions of these markers to reconstruct the 3d position of the headset. I would suspect using the IR markers on the oculus to track the user's head would be ideal. That way the user doesn't get motion sick from all the noise I suspect the kinect would have and use the kinect for tracking the rest of the body. I wouldn't expect the kinect tracking to be accurate/responsive enough to have a seamless interaction though.
You have many problems to solve.
In some VR systems you are only able to track the head and the hands. Three data points make it VERY difficult to work out a body position.
VR systems that use lights to track the data points have the vision problem. Hold a Sony move controller in front of you, perfect tracking. Hold it behind your back, no tracking.
Then what happens when someone wanders into the room. You have a headset on and can't see them, so you have no clue what's going on.
I am waiting for the first court case of someone smashing their TV, or worse their child, with a controller while wearing a headset.
VR is very much a hot topic at the moment, but don't get carried away with it. The technology isn't great and the potential for failure huge.
Seems feasible. The problem is: What if you want a game that consists of more than just one room?
It can be solved by placing two or more kinect, they can be linked on the same computer, and requested via their ID.
Three data points make it VERY difficult to work out a body position.
The location in 3D space will be made by using the Kinect Skeletal Tracking, which is designed for the tracking of a person in a room.
VR systems that use lights to track the data points have the vision problem. Hold a Sony move controller in front of you, perfect tracking. Hold it behind your back, no tracking.
It could be placed 4 Kinect, one per wall.
I am waiting for the first court case of someone smashing their TV
Of course it will not be a home solution.
