Jump to content

  • Log In with Google      Sign In   
  • Create Account


Member Since 24 Jul 2012
Offline Last Active Sep 14 2012 02:20 AM

Topics I've Started

Estimating surface normal from depth map and smoothing

23 August 2012 - 03:21 AM

Currently i'm doing some image processing on depth map (using OpenNI and XTion)
My depth map size is 320x240 (cannot make it 640x480 due to performance issue since we're developing a game)
The image processing should be real-time so it's fully written in fragment shader.
I generate normal map using simple cross product of 8 neighbouring pixels

Posted Image

Since the depth is noisy, i do a hack, so i calculate the 8 pixels like this

[source lang="cpp"]float step = _range/512.0; // 512.0 is tex sizefloat4 top = tex2D(_SubTex, float2(i.uv.x, i.uv.y - step));float4 right = tex2D(_SubTex, float2(i.uv.x + step, i.uv.y));//and so on....[/source]

i can adjust the _range variable and looking for farther pixels

Here's the result (raw depth map, now prepocessing at all)
Posted Image

2nd result (5x5 bilateral filter is applied first)

Posted Image

My approach to estimate normal is kinda stupid and i need suggestions to do it in a better way Posted Image
And also i'm considering to apply 5x5 median filter (i've tried 3x3 media filter but it's too small to remove noise) but it will be difficult to use such a huge 5x5 window due to implementation and performance issue

Kinect usermap smoothing using hq4x

26 July 2012 - 01:07 AM

Hi, right now i am doing a post-processing of kinect's depth map. This is, of course, for gaming purpose so i need everything runs in realtime.
The game itself will be in HD and as you can guess, the problem is kinect's depth map has row resolution and it wouldn't be good. So i decided to use hq4x to smooth the kinect's depth map since this filtering is very good on binary image.

Here's my steps:
  • My depth map resolution is only 320x240 (it's a sufficient resolution for gesture tracking, higher resolution may cause performance problem)
  • Segment user's body only (this is very easy using user mask from OpenNI)
  • I am using texture buffer and for speed consideration, the buffer should be power-of-two which is 512x512 (i don't want to cut my depth map so 512 is decided as the higher-nearest power-of-two).
  • Then the 320x240 depth image is downsampled to 80x60 so when it's applied on hq4x the resolution is back to 320x240 and it's fit to 512x512 texture buffer
  • Apply hq2x upsampling on 80x60 image
Here's my input:
Posted Image

And this is my output:
Posted Image

As you can see, hq4x does pretty well and the result is very good but there's an issue, the resulted image still has sharp curvatures on the edge.

What i want look like this:
Posted Image

Currently i am stuck with this problem and i'm still thinking how to improve the result. Actually, I have an idea to do morphological erosion (or dilation) first, anyway i need to do research again :)

Perhaps anyone here has another idea :)

Thanks !