Jump to content
  • Advertisement
Sign in to follow this  

Kinect skeleton based mesh deformation

This topic is 2211 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi I am currently working on a piece of software based in WPF 3D(cant use XNA or DirectX). It requires that I take a mesh and deform it using a kinect skeleton. The software has two areas, one where binding is performed and the other where deformation is performed. The basic idea is I take a snapshot of someones skeleton in an approriate bind pose. I position the mesh we want to deform in front of the skeleton and tell it to bind. 3D distance calculations determine what bones are connected to which vertices and their weighting are based on distance to the bone. So I now have a binding skeleton and our mesh weighted.

In the deforming stage I take this information and calculate the inverse matrices for the binding skeleton and apply this to the mesh. This deforms the mesh to skeleton unity. I then multiply the unity mesh by the current user skeleton matrices for who ever is in front of the kinect. The idea is that the mesh should bind to the user skeleton and we should have a mesh that is deformable by the user.

However I cant seem to get it right. There appears to be stretching of the mesh and rotation instability as I move the joints (causing sharp jumps).
So I have three basic methods BindBones, GenerateInverseBindingSkeletonTransformedMesh and DeformMesh. The first determines the weighting of the mesh and what vertices are connected to which bones, the second transforms the mesh we have by the inverse of the binding skeleton matrices and the third takes that mesh and transforms it using the current user skeleton.

I've attached a screenshot exhibiting the behaviour, the binding and deforming skeleton are exactly the same thanks to the dummy but the mesh clearly isnt the same. With smooth skinning switched on it goes nuts, with smooth skinning off(only one bone deforming) it looks nearly right. Has anyone got any idea of what this behaviour might be caused by? I can provide more detail if required.
Thanks, Brett

Share this post

Link to post
Share on other sites
Sign in to follow this  

  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!