• Advertisement
Sign in to follow this  

Tracking with Kinect and Oculus Rift character in-game

This topic is 871 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Do you think you can replace the mouse and keyboard with Kinect and Oculus Rift to move a character in a 3D environment, such as a reconstruction 1:1 of the room where you are in 3D? The reconstructed room will contain chairs and table in 3D, and the player wearing the Oculus Rift sees from the eyes of the character, and moving physically in the room moves his virtual alter-ego. Is it achievable? The sampling of the skeleton via IR could make crazy the character, making him "vibrate" or detect an unwanted movement.

Share this post


Link to post
Share on other sites
Advertisement

Wouldn't surprise me if this isn't already done. I have seen a demo that used something to track a person but I don't recall what that something was, it might have been Kinect. If you are interested in full room movement you should check out the HTC/Valve Vive. 

 

This is also very cool:

https://thevoid.com

They have all sorts of things going on in this and various effects to make people feel like they are there. Unfortunately I can't find the link now but I've also seen some university doing full moving VR using a motion capture type setup but that really is a bit beyond your average home solution. Still, it does show what's possible.

Share this post


Link to post
Share on other sites
I did some research around VR/augmented reality. I found that the best motion tracking generally used retro reflective markers with cameras that track the markers. Of course that is over most people's budget. I used a leap motion to try to track fingers connected to an oculus rift. It worked fairly well.

My specific research was trying to mark real world objects with tags (not my video, but that is pretty much how mine looked) so the computer could track them. That way when you pick up a block in the real world you can feel the weight and shape of the block and the computer could replace the block with anything in the VR world. I used this to make a chess game where you move physical pieces to play against a virtual opponent. The problem I ran into was knowing where your hand were in the virtual world. You mostly had to feel around blindly to play. That is why I tried using leap motion, but it would completely fail to track the hands once you picked anything up or even when your hands were near the chess pieces.

I haven't used knect but I suspect trying to use it for VR would result in either jumpy simulation or, using some sort of smoothing, would result in a laggy vr experience. Tracking a tag can be done much more accurately and much faster but requires more setup.

The second and third oculus that is coming out uses infrared lights on the headset as markers and a camera tracks the positions of these markers to reconstruct the 3d position of the headset. I would suspect using the IR markers on the oculus to track the user's head would be ideal. That way the user doesn't get motion sick from all the noise I suspect the kinect would have and use the kinect for tracking the rest of the body. I wouldn't expect the kinect tracking to be accurate/responsive enough to have a seamless interaction though.

Share this post


Link to post
Share on other sites

You have many problems to solve.

 

In some VR systems you are only able to track the head and the hands. Three data points make it VERY difficult to work out a body position.

 

VR systems that use lights to track the data points have the vision problem. Hold a Sony move controller in front of you, perfect tracking. Hold it behind your back, no tracking.

 

Then what happens when someone wanders into the room. You have a headset on and can't see them, so you have no clue what's going on.

 

I am waiting for the first court case of someone smashing their TV, or worse their child, with a controller while wearing a headset.

 

VR is very much a hot topic at the moment, but don't get carried away with it. The technology isn't great and the potential for failure huge.

Share this post


Link to post
Share on other sites

Seems feasible. The problem is: What if you want a game that consists of more than just one room?

 

It can be solved by placing two or more kinect, they can be linked on the same computer, and requested via their ID.

 


Three data points make it VERY difficult to work out a body position.

 

The location in 3D space will be made by using the Kinect Skeletal Tracking, which is designed for the tracking of a person in a room.

 


VR systems that use lights to track the data points have the vision problem. Hold a Sony move controller in front of you, perfect tracking. Hold it behind your back, no tracking.

 

It could be placed 4 Kinect, one per wall.

 


I am waiting for the first court case of someone smashing their TV

 

Of course it will not be a home solution.

Edited by Ordnas

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement