View more

View more

View more

### Image of the Day Submit

IOTD | Top Screenshots

### The latest, straight to your Inbox.

Subscribe to GameDev.net Direct to receive the latest updates and exclusive content.

# Kinect skeleton based mesh deformation

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

No replies to this topic

### #1Enfusion  Members

Posted 01 October 2012 - 06:12 AM

Hi I am currently working on a piece of software based in WPF 3D(cant use XNA or DirectX). It requires that I take a mesh and deform it using a kinect skeleton. The software has two areas, one where binding is performed and the other where deformation is performed. The basic idea is I take a snapshot of someones skeleton in an approriate bind pose. I position the mesh we want to deform in front of the skeleton and tell it to bind. 3D distance calculations determine what bones are connected to which vertices and their weighting are based on distance to the bone. So I now have a binding skeleton and our mesh weighted.

In the deforming stage I take this information and calculate the inverse matrices for the binding skeleton and apply this to the mesh. This deforms the mesh to skeleton unity. I then multiply the unity mesh by the current user skeleton matrices for who ever is in front of the kinect. The idea is that the mesh should bind to the user skeleton and we should have a mesh that is deformable by the user.

However I cant seem to get it right. There appears to be stretching of the mesh and rotation instability as I move the joints (causing sharp jumps).
So I have three basic methods BindBones, GenerateInverseBindingSkeletonTransformedMesh and DeformMesh. The first determines the weighting of the mesh and what vertices are connected to which bones, the second transforms the mesh we have by the inverse of the binding skeleton matrices and the third takes that mesh and transforms it using the current user skeleton.

I've attached a screenshot exhibiting the behaviour, the binding and deforming skeleton are exactly the same thanks to the dummy but the mesh clearly isnt the same. With smooth skinning switched on it goes nuts, with smooth skinning off(only one bone deforming) it looks nearly right. Has anyone got any idea of what this behaviour might be caused by? I can provide more detail if required.
Thanks, Brett

#### Attached Thumbnails

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.