Jump to content

  • Log In with Google      Sign In   
  • Create Account

We're offering banner ads on our site from just $5!

1. Details HERE. 2. GDNet+ Subscriptions HERE. 3. Ad upload HERE.


Estimating surface normal from depth map and smoothing


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
6 replies to this topic

#1 azer.darkblade   Members   -  Reputation: 139

Like
0Likes
Like

Posted 23 August 2012 - 03:21 AM

Hi,
Currently i'm doing some image processing on depth map (using OpenNI and XTion)
My depth map size is 320x240 (cannot make it 640x480 due to performance issue since we're developing a game)
The image processing should be real-time so it's fully written in fragment shader.
I generate normal map using simple cross product of 8 neighbouring pixels

Posted Image

Since the depth is noisy, i do a hack, so i calculate the 8 pixels like this

[source lang="cpp"]float step = _range/512.0; // 512.0 is tex sizefloat4 top = tex2D(_SubTex, float2(i.uv.x, i.uv.y - step));float4 right = tex2D(_SubTex, float2(i.uv.x + step, i.uv.y));//and so on....[/source]

i can adjust the _range variable and looking for farther pixels

Here's the result (raw depth map, now prepocessing at all)
Posted Image


2nd result (5x5 bilateral filter is applied first)

Posted Image

My approach to estimate normal is kinda stupid and i need suggestions to do it in a better way Posted Image
And also i'm considering to apply 5x5 median filter (i've tried 3x3 media filter but it's too small to remove noise) but it will be difficult to use such a huge 5x5 window due to implementation and performance issue

Edited by azer.darkblade, 23 August 2012 - 03:26 AM.


Sponsor:

#2 jefferytitan   Crossbones+   -  Reputation: 2242

Like
0Likes
Like

Posted 23 August 2012 - 06:00 AM

Yeah, I think picking an arbitrary range and doing a cross product is a little hokey. Posted Image How about calculating distance-weighted depths for each of the four corners used? This could be done by only sampling on the diagonals, or alternately by sampling whole quadrants.You may also want to pre-process using a median filter for spot removal.

A more accurate approach might be something like the below, although probably too slow for what you want:
http://www.pointclouds.org/documentation/tutorials/normal_estimation.php

#3 Seabolt   Members   -  Reputation: 633

Like
1Likes
Like

Posted 27 August 2012 - 10:20 AM

This may be fairly random, but I think you and I are working on the same project :) I've been doing the Kinect graphics for the 360 on a strikingly similar background ;)

The approach I used to get fairly good results was to just do a simple downsample to a half size target, leveraging bi-linear filtering. If you need to maintain the size of your normal map as you have it now, this may not work, since it thins out the appendages, but to our 120x90 image it looks great. I also had some really great results by applying a gaussian blur on top of everything.
Perception is when one imagination clashes with another

#4 azer.darkblade   Members   -  Reputation: 139

Like
0Likes
Like

Posted 27 August 2012 - 07:51 PM

This may be fairly random, but I think you and I are working on the same project Posted Image I've been doing the Kinect graphics for the 360 on a strikingly similar background ;)

The approach I used to get fairly good results was to just do a simple downsample to a half size target, leveraging bi-linear filtering. If you need to maintain the size of your normal map as you have it now, this may not work, since it thins out the appendages, but to our 120x90 image it looks great. I also had some really great results by applying a gaussian blur on top of everything.


Hahaha I see Posted Image , good luck with your 360 project. mine uses Unity
i did a little bit tweak on my bilateral filter and it looks better right now. yes you are right by downsampling and applying gaussian filter, the result is more consistent and less noise, right now i choose not to lose the detail and keep on 320x240 resolution instead. I got depth map and normal map, hey i can add SSAO effect now Posted Image

Edited by azer.darkblade, 28 August 2012 - 01:48 AM.


#5 azer.darkblade   Members   -  Reputation: 139

Like
1Likes
Like

Posted 28 August 2012 - 09:41 PM

It's me again,
Actually i still have one more issue. As you can see in the image below. My hand is in front of my chest but as as viewed on the normal calculation result the hand seems connected with my chest.

Posted Image

#6 jefferytitan   Crossbones+   -  Reputation: 2242

Like
1Likes
Like

Posted 28 August 2012 - 09:48 PM

I see that as a risk of applying general blurring. Personally I would selectively blur, e.g. do spot removal first and then only blur together depth values that are within a certain tolerance of each other. Realistically noise will usually only cause small errors, apart from weird outliers that spot removal could catch.

#7 Seabolt   Members   -  Reputation: 633

Like
0Likes
Like

Posted 29 August 2012 - 11:22 AM

It's true, I'd weight the blur with the difference between the depths.
Perception is when one imagination clashes with another




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS