• Advertisement
  • entries
  • comments
  • views

Kinect Programming with Direct3D 11

Sign in to follow this  


Those of you who follow the Hieroglyph 3 development on its codeplex page may have noticed that I have added a new sample project: KinectPlayground. Hieroglyph 3 was created as a Direct3D 11 framework, and all of its samples up until now have all been used to demonstrate some technique or algorithm that uses D3D11 features - so it may be somewhat of a surprise to see a Kinect based sample. There are a few simple reasons for this addition:

  1. The Kinect will be a very big deal in the next couple years. It has been successful so far, but once it hits for Windows, it will really become a huge device with major numbers of users.
  2. I am a big fan of computer vision algorithms, and the Kinect provides some things that aren't as easy to get with other devices. It has a unique mix of color and depth input images, which can make for some interesting new algorithms
  3. The GPU is the perfect place to perform some image processing algorithms, so mapping the Kinect input data into a configuration that can be used by Direct3D 11 is a good idea.

Many of my favorite courses in my undergrad and Masters degrees were related to image and/or signal processing, so the Kinect provides a great opportunity for me to exercise that particular itch. This sample provides a fairly simple introduction to working with the Kinect by simply mapping the color and depth buffer contents into appropriately created Texture2D resources. I have already seen a forum posts asking about how to properly map the data from the Kinect to D3D11 GPU resources, so I built up the sample in such a way that should be fairly easily portable to another framework.

My intentions to work with these resources is to provide some new functionality outside of the standard human pose acquisition. The Kinect SDK provides methods for obtaining the human poses of those people that are within view of the Kinect cameras. While I am sure I will be using this too at some point, I want to push the limit on what people are doing with the raw depth and color images. Certainly more info will follow on exactly what I mean down the road.

I am also going to be doing some work with OpenCV, which is a computer vision library that is available with the BSD license. While I normally like implementing any algorithms that I use myself, I'm also realistic about how much time I will have to get something done. I'll choose my battles the best way that I can, and try to use my time for implementing features that use D3D11 over general algorithms that are already available (via OpenCV). Things like calibrating cameras shouldn't be the focus of my work :)

Due to the fact that working with the Kinect requires another SDK to be installed, plus it has to be running on Windows 7 or 8, I have been careful to ensure that the dependencies to the Kinect SDK are isolated to the sample applications. Instead of building it into the engine itself, I am adding the Kinect stuff for at the sample program level. The result is that if you don't have the appropriate pre-requisites, then just the Kinect sample will fail to compile while the others are still build-able and usable.

So if you are a Kinect fan and want to see some particular algorithms, features, or applications, please post a comment either here, or in the Hieroglyph 3 discussion pages!
Sign in to follow this  


Recommended Comments


Really interested to see how this project turns out! I also notice that you mentioned that you need to be running Windows 7 or 8, and I don't think that is the case for the kinect in general. Last week a friend at work had one running on linux. It was probably running via ROS, which has [url="http://www.ros.org/wiki/kinect"]support for the kinect and some openCV examples[/url] apparently. I thought you might find it interesting.


Share this comment

Link to comment
Hi, thanks for the info. I'm using the official Kinect Beta SDK, which does require Windows 7 or 8. There are a couple other SDKs and/or drivers out there that work on all OS's, but I wanted to be in the clear with any work that I do on the project regarding licensing. Plus the skeletal tracking that is provided with the Microsoft SDK is significantly better than most other implementations that I have seen....

The ROS project is pretty cool - some of the guys I work with (in a different division actually, but still in contact) are doing lots of robotics work with ROS, so I am familiar with the overview concepts. Even so, I haven't touched any of the actual code myself, so it could well be worth checking into sometime down the road. Thanks again for the info!

Share this comment

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Advertisement