• FEATURED

View more

View more

View more

### Image of the Day Submit

IOTD | Top Screenshots

### The latest, straight to your Inbox.

Subscribe to GameDev.net Direct to receive the latest updates and exclusive content.

# Help with GPU Pro 5 Hi-Z Screen Space Reflections

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

78 replies to this topic

### #61LukasBanana  Members

Posted 04 November 2014 - 10:45 AM

Ok I think I finally understood how the "intersectCellBoundary" function works.

But could it be, that the crossStep and crossOffset should be computed after the ray direction was 'normalized'?

// get the cell cross direction and a small offset to enter the next cell when doing cell crossing
float2 crossStep = float2(v.x >= 0.0f ? 1.0f : -1.0f, v.y >= 0.0f ? 1.0f : -1.0f);
float2 crossOffset = float2(crossStep.xy * HIZ_CROSS_EPSILON.xy);
crossStep.xy = saturate(crossStep.xy);

// scale vector such that z is 1.0f (maximum depth)
float3 d = v.xyz / v.z;


Maybe this?

// scale vector such that z is 1.0f (maximum depth)
float3 d = v.xyz / v.z;

// get the cell cross direction and a small offset to enter the next cell when doing cell crossing
float2 crossStep = float2(d.x >= 0.0f ? 1.0f : -1.0f, d.y >= 0.0f ? 1.0f : -1.0f);
float2 crossOffset = float2(crossStep.xy * HIZ_CROSS_EPSILON.xy);
crossStep.xy = saturate(crossStep.xy);


Because when v.z is negative, d.x and d.y will be fliped.

I still don't get correct results, but it seems to be more logical for me right now.

UPDATE:

I finally found a problem: Currently the ray-marching through the Hi-Z buffer does not work correctly for NPOT (non-power-of-two) resolutions.

With the ray-marching visualization I noticed that the 'cells' where not located correctly.

As you can see in the screenshot below I now use temporarily a resolution of 512x512, where the cells are correct:

I tried to modify the "GetCellCount" function to always have even 'counts':

vec2 GetCellCount(vec2 size, float level)
{
return floor(size / (level > 0.0 ? exp2(level) : 1.0));
//  return       size / (level > 0.0 ? exp2(level) : 1.0) ;
}


But that didn't work.

And I still have wrong results when the ray dir Z is negative. But I stay tuned :-)

Edited by LukasBanana, 04 November 2014 - 12:22 PM.

### #62WFP  Members

Posted 04 November 2014 - 03:21 PM

Hey Lukas,

Regarding the NPOT restriction, this was a problem we worked through earlier in the thread. I'm traveling right now, so don't have time to read back through or do a big post here, but read back over the thread because I'm almost positive I posted a solution for that somewhere in there. If I remember correctly, I changed the way I was doing offsets while building out the various buffers. Let me know if you get it fixed. I'll re-read the thread when I get time and try to find the exact post if I get some time later and you haven't solved it yet.

Edit: Look at post #24. I've tested that on both an NVidia and laptop ATI cards so it should work regardless of which you use.

Thanks,
WFP

### #63LukasBanana  Members

Posted 05 November 2014 - 11:47 AM

I used positive offsets for the Hi-Z Buffer generation from the very beginning of my implementation.

This is relevant part of my GLSL shader's main function to generate the Hi-Z maps:

/* Fetch texels from current MIP LOD */
ivec2 coord = ivec2(gl_FragCoord.xy) * ivec2(2);

vec4 texels[2];
texels[0].rg = texelFetch(tex, coord, 0).rg;
texels[0].ba = texelFetch(tex, coord + ivec2(1, 0), 0).rg;
texels[1].rg = texelFetch(tex, coord + ivec2(0, 1), 0).rg;
texels[1].ba = texelFetch(tex, coord + ivec2(1, 1), 0).rg;

/* Down-sample texels */
float minZ = min(min(texels[0].r, texels[0].b), min(texels[1].r, texels[1].b));
float maxZ = max(max(texels[0].g, texels[0].a), max(texels[1].g, texels[1].a));


BTW: I don't pass the current MIP level or any other data via constant buffer.

I just bind the current MIP level to render in and clamp the MIP level to sample from:

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_BASE_LEVEL, mipLevel - 1);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAX_LEVEL, mipLevel - 1);


But that's not the solution for my problem.

I think the relation between the 'cells' and the 'texels', sampled from the Hi-Z buffer, is currently not correct for NPOT resolutions.

I presume the "intersectCellBoundary" function fails in this case when I pass the "cellCount" parameter.

### #64WilliamHamilton  Members

Posted 14 November 2014 - 02:38 PM

Hello,

That is very cool to find other programmers with the same issues.

Currently I am working on 100% reflective surface, now I have similar result like @WFP:

http://www.gamedev.net/topic/658702-help-with-gpu-pro-5-hi-z-screen-space-reflections/#entry5173175

Strairs, noise... I cannot fix it like you.

For me I have lot of contradiction of the original paper for exemple:

Pseudo code on p163 use min and boundary plane and on implementation p174 he only call function named "getMinDepthPlane".

He test both min and boundary plane without ++ or -- on Mips not on p174 implementation.

On p160 he talk about importance of perspective interpolation intersectDepthPlane should be more complexe than simple return o + d*t; we need perspective interpolation on Depth buffer.

Other open question did you know how we can test if we are on "Failure cases":

http://bartwronski.com/2014/01/25/the-future-of-screenspace-reflections/

For failure case 1 it is pretty easy just test the UV but how can we do the test for case 2 and 3?

It is possible to have someone can share full code for hiZTracing? Or/And how to compute the parameters of HiZ?

Thanks

### #65LukasBanana  Members

Posted 15 November 2014 - 03:34 PM

@WilliamHamilton:

On p160 he talk about importance of perspective interpolation intersectDepthPlane should be more complexe than simple return o + d*t; we need perspective interpolation on Depth buffer.

Sure, this function is really that simple. He explains we can simply linear-interpolate the post-projected Z, because we interpolate it in (post-projected) screen space.

Figure 4.10 on p.160 is helpful here.

It is possible to have someone can share full code for hiZTracing? Or/And how to compute the parameters of HiZ?

I can't do that right now, sorry. It's part of my bachelor thesis. When my work is done (expected next year), it will be possibly open-source.

@all:

I finally resolved the staircase artifacts with NPOT (non-power-of-two) resolutions!

Thanks WFP, your hint about the Hi-Z MIP generation was very helpful :-), but I have a slightly different solution.

Here I made a sketch for a single Hi-Z MIP generation pass (example shows generation of the MIP size 2x2 from the previous size 5x4):

Instead of always sampling with the offsets (0, 0), (1, 0), (0, 1), (1, 1), I use an offset which is either 1 or 2.

Pseudocode:

tex.Load(coord + int2(0, 0));


As you can see I don't use "SampleLevel" (or "textureLodOffset" in GLSL), instead I use "Load" (or "texelFetch" in GLSL).

On the C++ side I update this offset before every MIP-Level generation pass:

// Check for width and height if it's even or odd.
constantBuffer.offset.x = (upperMIPLevelSize.width  % 2 == 0 ? 1 : 2);
constantBuffer.offset.y = (upperMIPLevelSize.height % 2 == 0 ? 1 : 2);


This works pretty good for me. Here is a comparision screenshot with my previous and current solution:

This is a 800x600 resolution but I also tested totally flubbed sizes like 812x617

Hope this helps someone :-)

Greetings,

Lukas

### #66WFP  Members

Posted 16 November 2014 - 09:22 PM

Hi WilliamHamilton,

Towards the end of the chapter, the author actually gives a few hints and examples to handle failure-type cases.  Here's the code I currently have in to fade rays as they become unusable, which is basically right from the book.  It could use a little touching up, but should get you on the right track.

        // fade pointing towards camera
float3 reflectVS = reflect(toPositionVS, normalVS);

// fade rays close to screen edge
float2 boundary = abs(raySS.xy - float2(0.5f, 0.5f)) * 2.0f;

// needs to be updated to use the cb_maximumDistance as part of its calculation
float traveled = distance(raySS.xy, positionSS.xy);



If you'd like help with other parts of your implementation, please post it here and I'll take a look at it when I get a moment to see if anything jumps out at me.

LukasBananas,

Glad to see you were able to resolve your step artifacts.  Nice job!

Thanks,

WFP

### #67WilliamHamilton  Members

Posted 17 November 2014 - 11:58 AM

I am more interested by your implementation of HiTrace.

My implementation (majority of the code is based on the implementation of you guy, mix):

mainPS:

    // Get Data
float4 normal = float4( NormalTexture.Sample( NormalTextureSamp, uv ).xyz,             1.f );

// Early discard: Remove the sky and surfaces without normals
if ( ( dot( normal, float3( 1.f, 1.f, 1.f ) ) ) <= 0.f )
return 0;

float4 color  = ColorTextureTexture.Sample( ColorTextureSamp, uv );
float4 depth  = float4( DepthTexture.Sample( DepthTextureSamp, uv ).xxx,               1.f );
float4 vis    = float4( MaterialTexture.Sample( MaterialTextureSamp, uv ).xyz,         1.f );
float4 hiZ    = float4( HiZDepthTexture.Sample( HiZDepthTextureSamp, uv ).xyz,         1.f );

// World Normal To ViewSpace Normal
float3 viewNormal = normalize( mul( NormalFromGNormal( normal.xyz ), ( float3x3 )ViewMatrix ) );

float usedDepth = depth.z; // ( min, max, depth )

// UV Space To Clip Space
float4 projPos = float4( UVToClipXY( uv.xy ), usedDepth, 1.0 );

// Clip Space to View Space
projPos = mul( projPos, InvProjectionMatrix );

float3 viewPos = projPos.xyz/projPos.w;

float3 viewDir = normalize( viewPos );

float3 viewReflect = normalize( reflect( viewDir, viewNormal ) );

// Clip Plane result
float4 screenReflectPos = mul( float4( viewPos + viewReflect, 1.0 ), ProjectionMatrix );

/*
if ( abs( screenReflectPos.w ) < 0.001f )
return 0;
*/

screenReflectPos.xyz /= screenReflectPos.w;

// Back to UV Space
screenReflectPos.xy = ClipXYToUV( screenReflectPos.xy );

float3 screenPos = float3( uv, usedDepth );
// Operation on UV Space
float3 screenReflect = normalize( screenReflectPos.xyz - screenPos );

if ( screenReflect.z < 0.f )
return 0.f;

float4 newUV = hiZTrace( screenPos, screenReflect );

if ( newUV.x < 0.f || newUV.y < 0.f || newUV.x > 1.f || newUV.y > 1.f )
return 0.f;

/*
float4 foundPos = float4( float2( -1.0, 1.0 ) + newUV.xy*float2( 2.0, -2.0 ), newUV.z, 1.0 );
foundPos = mul( foundPos, InvProjectionMatrix );
float3 viewFoundPos = foundPos.xyz/foundPos.w;

float4 foundPosD = float4( float2( -1.0, 1.0 ) + newUV.xy*float2( 2.0, -2.0 ), depth.x, 1.0 );
foundPosD = mul( foundPos, InvProjectionMatrix );
float3 viewFoundPosD = foundPosD.xyz/foundPosD.w;
*/

float4 reflectedColor = ColorTextureTexture.Sample( ColorTextureTextureSamp, newUV.xy );

return reflectedColor;


And my HiTrace implementation:

float3 intersectDepthPlane( float3 o, float3 d, float t )
{
return o + d*t;
}

float2 getCell(float2 ray, float2 cellCount)
{
return floor( ray*cellCount );
}

float3 intersectCellBoundary(float3 o, float3 d, float2 cellIndex, float2 cellCount, float2 crossStep, float2 crossOffset)
{
float2 index = cellIndex + crossStep;
index /= cellCount;
index += crossOffset;
float2 delta = index - o.xy;
delta /= d.xy;
float t = min( delta.x, delta.y );
return intersectDepthPlane( o, d, t );
}

float2 getCellCount(float level, float rootLevel)
{
float2 div = ( level == 0.0f ? 1.0f : exp2( level ) );
return floor( ViewportSize.xy/div );
}

float3 getMinimumDepthPlane( float2 ray, float level, float rootLevel )
{
[branch]
if ( level == 0 )
return DepthTexture.tex.SampleLevel( DepthTexture.samp, ray.xy, 0.f ).xxx;
else
return HiZDepthTexture.tex.SampleLevel( HiZDepthTexture.samp, ray.xy, level ).xyz;
}

bool crossedCellBoundary( float2 cellIdxOne, float2 cellIdxTwo )
{
return floor( cellIdxOne.x ) != floor( cellIdxTwo.x ) || floor( cellIdxOne.y ) != floor( cellIdxTwo.y );
}

float4 hiZTrace( float3 p, float3 v )
{
const float rootLevel = HIZ_MAX_LEVEL - 1.0f;

float level = HIZ_START_LEVEL;

uint iterations = 0u;

float3 d = v.xyz / v.z;

float2 crossStep   = float2( d.x >= 0.f ? 1.0f : -1.0f, d.y >= 0.f ? 1.0f : -1.0f );
float2 crossOffset = float2( crossStep.xy*HIZ_CROSS_EPSILON.xy );
crossStep.xy       = saturate( crossStep.xy );

float4 ray = float4( p.xyz, 0.f );

float3 o = intersectDepthPlane( p, d, -p.z );

float2 hiZSize = getCellCount( level, rootLevel );
float2 rayCell = getCell( ray.xy, hiZSize );

ray.xyz = intersectCellBoundary( o, d, rayCell.xy, hiZSize.xy, crossStep.xy, crossOffset.xy );

float cumul = 0.f;

[loop]
while ( level >= HIZ_START_LEVEL && iterations < MAX_ITERATIONS )
{
crossStep    = float2( d.x >= 0.f ? 1.0f : -1.0f, d.y >= 0.f ? 1.0f : -1.0f );
crossOffset  = float2( crossStep.xy*HIZ_CROSS_EPSILON.xy );
crossStep.xy = saturate( crossStep.xy );

const float2 cellCount  = getCellCount( level, rootLevel );
const float2 oldCellIdx = getCell( ray.xy, cellCount );

float3 zMinMax0 = getMinimumDepthPlane( ray.xy, level, rootLevel );
//float3 zMinMax1 = getMinimumDepthPlane( ray.xy, min( level + 1, HIZ_MAX_LEVEL ), rootLevel );

//float3 zClosest = min( zMinMax0, zMinMax1 );
float3 zClosest = zMinMax0;

float3 tmpRay = intersectDepthPlane( o, d, max( ray.z, zClosest.z ) );
// get the new cell number as well
const float2 newCellIdx = getCell( tmpRay.xy, cellCount );

[branch]
if ( crossedCellBoundary( oldCellIdx, newCellIdx ) )
{
tmpRay = intersectCellBoundary( o, d, oldCellIdx, cellCount.xy, crossStep.xy, crossOffset.xy );
level = min( HIZ_MAX_LEVEL, level + 1.f );
}
else
{
level = max( level - 1.f, 0 );
}

ray.xyz = tmpRay.xyz;

++iterations;
}

return float4( ray.xyz, iterations );
}


I try with power of 2 back Buffer but I still have noise or stairs depend of steps...

If you see something wrong on my code I will be happy

I am still working on "RayTracing" without blurring...

Thanks

### #68WilliamHamilton  Members

Posted 17 November 2014 - 12:10 PM

If you can't share HiTrace could you share a screen of each step to compare?

viewDir, screenReflected...? On ImageSpace?

### #69WFP  Members

Posted 17 November 2014 - 06:56 PM

Hi WilliamHamilton,

There are a few noticeable differences I see in your implementation that I do not have in mine.

First, I let getCellCount return whatever the value is without taking the floor() of it.  When I add floor like your implementation, I get a few block/stair artifacts in certain situations.  I think it's okay to let it return the real value, including fractional parts.

For getMinimumDepthPlane, I'm not sure why you are returning a float3.  At most, I think you would want a float2 (x for min and y for max from the hiz-buffer).  In the code you posted, you're using the z-component of that value, so what are you storing there?  If you look at the implementations for constructing the HiZ Buffer in this thread, I think you'll find that you really only need two values.

In crossedCellBoundary, I don't use floor() for any of my comparison values and it seems to work fine.  It may not be an error, but you can probably remove it - test with and without to see what works.

You can also move your crossStep and crossOffset calculations out of the loop, as what you posted will generate the same values for them every iteration.

What values are you using for HIZ_START_LEVEL, HIZ_STOP_LEVEL, HIZ_MAX_LEVEL, HIZ_CROSS_EPSILON, and MAX_ITERATIONS?

Thanks,

WFP

### #70WilliamHamilton  Members

Posted 18 November 2014 - 11:14 AM

@WFP:

Indeed without the floor on my GetCell that fix the issue of stairs.

GetMinDepthPlane give me min/max and depth at (0,0) offset. I use this value for debug & test. But I test with the min and max now and I got the same result with:

float3 tmpRay = intersectDepthPlane( o, d, max( ray.z, zMinMax.y ) ); // (or zMinMax0.x)

If we don't use a floor so we compare directly floats and that work ok !

For now Start and Stop are 1 or 2 CrossEpsilon like you I think: PixelSize.xy*exp2( HIZ_START_LEVEL + 1.f );

And Max iteration sadly it is between 512 and 1024.

I try to do like you

http://www.gamedev.net/topic/658702-help-with-gpu-pro-5-hi-z-screen-space-reflections/page-3#entry5187536

To have the "fix" for the "A Shape" like feets. I use directly the pixels with Load but that break the reflection and I have lot of self collision and when I use:

float3 tmpRay = intersectDepthPlane( o, d, max( ray.z, max( zMinMax.x, zMinMax.y ) ) ); // First Min to Max

Instead, my reflection back (but with more artefact than before) it is possible my Z is not like your. Me I have 0 near and 1 far... (Sorry I can share screens).

The Switch to the Load instead SampleLevel look like a floor no?

Last question

For the fading I am not sure about the fading for travel distance. A ray can travel the distance he want, for exemple to reflect the sky, I am not sure is good information to remove the "Ray Hit" of "Projection Shadow".

I was try with iteration count, with ray.z VS real depth... I don't found (yet) a good information for that.

Thanks

William

### #71WFP  Members

Posted 18 November 2014 - 07:07 PM

Hi WilliamHamilton,

Your formula to get tmpRay is wrong based on what you posted.  You are using max() twice when it should really be

float3 tmpRay = intersectDepthPlane(o, d, min(max(ray.z, minZ.r), minZ.g));


This ensures the value is confined between the minimum and maximum of the hi-z buffer values for that pixel.

I'm using Load for every texture retrieval in my ray tracing steps, so that shouldn't be the issue as long as you are calculating the load position correctly.

Hope that helps,

WFP

### #72WilliamHamilton  Members

Posted 20 November 2014 - 08:50 AM

I will try to continu on this way thanks.

About performances did you reach the same results as the author? Did you use regular Pixel Shader or Compute? With which hardware, resolution, tech (OGL, D)...?

How many pass did you have?

- HiZ

- PreIntegration

- PreConvolution

- RayTracing + Cone Tracing ? (on 1 or 2 steps ?)

- Final Rendering

4, 5, 6 ?

### #73WFP  Members

Posted 20 November 2014 - 08:09 PM

Hi WilliamHamilton,

Yes the performance I'm getting out of the effect in most cases is pretty good.  I set my maximum iterations to 128, but usually the result is found in under 16 steps.  I'm doing the entire effect in pixel shaders only.  I'm using an NVIDIA GTX 760 on my desktop and an AMD 5770M on my laptop.  I run my test scene at 1536x864, but mostly just to make sure my effects can handle awkward resolutions.  My renderer uses DirectX 11 only.

The major passes you listed are all the ones I have in this effect (5).  Of course, may of those have several passes to them as the downsample into smaller mip channels, but you listed all the big steps.  I do split ray tracing and cone tracing into two separate passes.  This is mainly because I've been meaning to go back and interject a pass between them to do some extra processing on the ray tracing buffer.  This would be things like filling in small gaps of missed rays and maybe adding temporal stability, etc.

Thanks,

WFP

### #74LukasBanana  Members

Posted 27 January 2015 - 07:40 AM

I'm again working on this effect and I'm stuck with the hidden-geometry problem.

How do you detect in the HiZRayTrace procedure if an intersection is invalid because of hidden geometry?

Here is a screenshot where the problem is clearly visible:

At the marked areas there should actually no reflection. How can I solve this?

Moreover the following code part for ray traversal behind geometry does not work for me:

// Does not work for me: I get lots of artifacts and the ray traversal is not correct
vec3 tmpRay = IntersectDepthPlane(rayOrigin, rayDir, clamp(rayPos.z, zMinMax.r, zMinMax.g)); // MIN/MAX

// Only the minimal Z version works for me:
vec3 tmpRay = IntersectDepthPlane(rayOrigin, rayDir, max(rayPos.z, zMinMax.r)); // MIN


### #75WFP  Members

Posted 31 January 2015 - 09:28 AM

Hi LukasBanana,

If I'm looking at your scene correctly, it looks like those reflections are from rays that are traveling back towards the camera.  I disallow such cases in my scene, and honestly between fall back methods like parallax corrected cube maps, I don't think I miss those reflections all that much (i.e., a player wouldn't notice them, I don't think).  If you want to disallow rays traveling back towards the camera, you can use a dot product in the ray tracing step to detect and early exit if the ray is pointing backwards.  If you absolutely want rays to be able to travel backwards, you'll have to do some additional processing for cases like what you're experiencing.  It's been a while since I've read the chapter, so I can't remember if the author gave tips for this or not, but I think earlier in this thread someone else had rays coming back towards the camera, so maybe something in one of those posts would be helpful?  I think it was jgrenier, so maybe look over his posts, but again I'm not positive.

Hope you get it straightened out

Edited by WFP, 31 January 2015 - 09:28 AM.

### #76LukasBanana  Members

Posted 03 February 2015 - 06:22 AM

Hi, the red rectangle on the left marks indeed rays moving towards the camera.

I do this with a linear ray march. But the rectangle on the right shows the rays, computed with Hi-Z ray tracing.

Yasin Uludag talks about a way to travel rays behind geometry, but he does not explain it in detail.

Only that a min- and max Hi-Z buffer is required, and that it's quite complicated.

Anyway I won't spent much more time on this problem before I complete my Bachelor thesis.

Greetings,

Lukas

### #77WFP  Members

Posted 03 February 2015 - 09:42 AM

Hi Lukas,
I don't have my code in front of me at the moment, but a few things I can think of.

What format is your HiZ buffer? I'm storing scene depth in a DXGI_D32_FLOAT format, so use DXGI_FORMAT_R32G32 for my min/max HiZ buffer to make sure I can get the depth as accurate as possible.

Also, what MIP level (HiZ start) are you starting at and what is the highest level you let the effect run to (HiZ stop)?

Tracing behind objects should work in a lot of cases if you have the buffer constructed properly and have followed the code in this thread. There are still cases where it will fail, but that's always going to be the case for screen space effects where you're trying to reconstruct data that isn't available.

-WFP

### #78LukasBanana  Members

Posted 04 February 2015 - 03:31 AM

I'm using R32G32 Float as well, and the min and max Z values are available to me in the final shader stage.

I start with the MIP level 2 and I stop when the MIP level reached 0.

### #79WFP  Members

Posted 04 February 2015 - 07:38 PM

Hi Lukas,

The best I can tell you right now are the values I'm using for my setup:

static const float HIZ_START_LEVEL = 0.0f; // always start <= HIZ_STOP_LEVEL
static const float HIZ_STOP_LEVEL = 0.0f; // set higher to use less iterations - quality vs performance tradeoff
static const float HIZ_MAX_LEVEL = float(cb_mipCount);
static const float2 HIZ_CROSS_EPSILON = float2(texelWidth, texelHeight) * 0.5f; // just needs to make sure in the next pixel, not skipping them entirely
static const uint MAX_ITERATIONS = 128u; // a good balance of quality vs performance - most common cases will not use anywhere near this full value
static const float2 hiZSize = cb_screenSize / exp2(HIZ_START_LEVEL);


I know you've said you can't share your code until your thesis is complete, but unfortunately I can't help much more without being able to see what you've got so far.  If something changes, definitely post your code here and I'd be happy to review it.

Thanks,

WFP

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.