View more

View more

View more

### Image of the Day Submit

IOTD | Top Screenshots

### The latest, straight to your Inbox.

Subscribe to GameDev.net Direct to receive the latest updates and exclusive content.

Sign up now

# Estimating surface normal from depth map and smoothing

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

6 replies to this topic

### #1azer.darkblade  Members

Posted 23 August 2012 - 03:21 AM

Hi,
Currently i'm doing some image processing on depth map (using OpenNI and XTion)
My depth map size is 320x240 (cannot make it 640x480 due to performance issue since we're developing a game)
The image processing should be real-time so it's fully written in fragment shader.
I generate normal map using simple cross product of 8 neighbouring pixels

Since the depth is noisy, i do a hack, so i calculate the 8 pixels like this

[source lang="cpp"]float step = _range/512.0; // 512.0 is tex sizefloat4 top = tex2D(_SubTex, float2(i.uv.x, i.uv.y - step));float4 right = tex2D(_SubTex, float2(i.uv.x + step, i.uv.y));//and so on....[/source]

i can adjust the _range variable and looking for farther pixels

Here's the result (raw depth map, now prepocessing at all)

2nd result (5x5 bilateral filter is applied first)

My approach to estimate normal is kinda stupid and i need suggestions to do it in a better way
And also i'm considering to apply 5x5 median filter (i've tried 3x3 media filter but it's too small to remove noise) but it will be difficult to use such a huge 5x5 window due to implementation and performance issue

Edited by azer.darkblade, 23 August 2012 - 03:26 AM.

### #2jefferytitan  Members

Posted 23 August 2012 - 06:00 AM

Yeah, I think picking an arbitrary range and doing a cross product is a little hokey. How about calculating distance-weighted depths for each of the four corners used? This could be done by only sampling on the diagonals, or alternately by sampling whole quadrants.You may also want to pre-process using a median filter for spot removal.

A more accurate approach might be something like the below, although probably too slow for what you want:
http://www.pointclouds.org/documentation/tutorials/normal_estimation.php

### #3Seabolt  Members

Posted 27 August 2012 - 10:20 AM

This may be fairly random, but I think you and I are working on the same project I've been doing the Kinect graphics for the 360 on a strikingly similar background ;)

The approach I used to get fairly good results was to just do a simple downsample to a half size target, leveraging bi-linear filtering. If you need to maintain the size of your normal map as you have it now, this may not work, since it thins out the appendages, but to our 120x90 image it looks great. I also had some really great results by applying a gaussian blur on top of everything.
Perception is when one imagination clashes with another

### #4azer.darkblade  Members

Posted 27 August 2012 - 07:51 PM

This may be fairly random, but I think you and I are working on the same project I've been doing the Kinect graphics for the 360 on a strikingly similar background ;)

The approach I used to get fairly good results was to just do a simple downsample to a half size target, leveraging bi-linear filtering. If you need to maintain the size of your normal map as you have it now, this may not work, since it thins out the appendages, but to our 120x90 image it looks great. I also had some really great results by applying a gaussian blur on top of everything.

Hahaha I see , good luck with your 360 project. mine uses Unity
i did a little bit tweak on my bilateral filter and it looks better right now. yes you are right by downsampling and applying gaussian filter, the result is more consistent and less noise, right now i choose not to lose the detail and keep on 320x240 resolution instead. I got depth map and normal map, hey i can add SSAO effect now

Edited by azer.darkblade, 28 August 2012 - 01:48 AM.

### #5azer.darkblade  Members

Posted 28 August 2012 - 09:41 PM

It's me again,
Actually i still have one more issue. As you can see in the image below. My hand is in front of my chest but as as viewed on the normal calculation result the hand seems connected with my chest.

### #6jefferytitan  Members

Posted 28 August 2012 - 09:48 PM

I see that as a risk of applying general blurring. Personally I would selectively blur, e.g. do spot removal first and then only blur together depth values that are within a certain tolerance of each other. Realistically noise will usually only cause small errors, apart from weird outliers that spot removal could catch.

### #7Seabolt  Members

Posted 29 August 2012 - 11:22 AM

It's true, I'd weight the blur with the difference between the depths.
Perception is when one imagination clashes with another

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.