Jump to content

  • Log In with Google      Sign In   
  • Create Account

We need your feedback on a survey! Each completed response supports our community and gives you a chance to win a $25 Amazon gift card!






Kinect Programming with Direct3D 11

Posted by Jason Z, 18 December 2011 · 1,183 views

Those of you who follow the Hieroglyph 3 development on its codeplex page may have noticed that I have added a new sample project: KinectPlayground. Hieroglyph 3 was created as a Direct3D 11 framework, and all of its samples up until now have all been used to demonstrate some technique or algorithm that uses D3D11 features - so it may be somewhat of a surprise to see a Kinect based sample. There are a few simple reasons for this addition:

  • The Kinect will be a very big deal in the next couple years. It has been successful so far, but once it hits for Windows, it will really become a huge device with major numbers of users.
  • I am a big fan of computer vision algorithms, and the Kinect provides some things that aren't as easy to get with other devices. It has a unique mix of color and depth input images, which can make for some interesting new algorithms
  • The GPU is the perfect place to perform some image processing algorithms, so mapping the Kinect input data into a configuration that can be used by Direct3D 11 is a good idea.
Many of my favorite courses in my undergrad and Masters degrees were related to image and/or signal processing, so the Kinect provides a great opportunity for me to exercise that particular itch. This sample provides a fairly simple introduction to working with the Kinect by simply mapping the color and depth buffer contents into appropriately created Texture2D resources. I have already seen a forum posts asking about how to properly map the data from the Kinect to D3D11 GPU resources, so I built up the sample in such a way that should be fairly easily portable to another framework.

My intentions to work with these resources is to provide some new functionality outside of the standard human pose acquisition. The Kinect SDK provides methods for obtaining the human poses of those people that are within view of the Kinect cameras. While I am sure I will be using this too at some point, I want to push the limit on what people are doing with the raw depth and color images. Certainly more info will follow on exactly what I mean down the road.

I am also going to be doing some work with OpenCV, which is a computer vision library that is available with the BSD license. While I normally like implementing any algorithms that I use myself, I'm also realistic about how much time I will have to get something done. I'll choose my battles the best way that I can, and try to use my time for implementing features that use D3D11 over general algorithms that are already available (via OpenCV). Things like calibrating cameras shouldn't be the focus of my work :)

Due to the fact that working with the Kinect requires another SDK to be installed, plus it has to be running on Windows 7 or 8, I have been careful to ensure that the dependencies to the Kinect SDK are isolated to the sample applications. Instead of building it into the engine itself, I am adding the Kinect stuff for at the sample program level. The result is that if you don't have the appropriate pre-requisites, then just the Kinect sample will fail to compile while the others are still build-able and usable.

So if you are a Kinect fan and want to see some particular algorithms, features, or applications, please post a comment either here, or in the Hieroglyph 3 discussion pages!




Hi,

Really interested to see how this project turns out! I also notice that you mentioned that you need to be running Windows 7 or 8, and I don't think that is the case for the kinect in general. Last week a friend at work had one running on linux. It was probably running via ROS, which has support for the kinect and some openCV examples apparently. I thought you might find it interesting.

-Josh
Cool stuff Posted Image
Hi, thanks for the info. I'm using the official Kinect Beta SDK, which does require Windows 7 or 8. There are a couple other SDKs and/or drivers out there that work on all OS's, but I wanted to be in the clear with any work that I do on the project regarding licensing. Plus the skeletal tracking that is provided with the Microsoft SDK is significantly better than most other implementations that I have seen....

The ROS project is pretty cool - some of the guys I work with (in a different division actually, but still in contact) are doing lots of robotics work with ROS, so I am familiar with the overview concepts. Even so, I haven't touched any of the actual code myself, so it could well be worth checking into sometime down the road. Thanks again for the info!

December 2014 »

S M T W T F S
 123456
78910111213
14151617181920
21222324252627
28 29 3031   

Recent Comments

Recent Comments

PARTNERS