Sign in to follow this  
RPTD

GLSL Clipping and gl_FragDepth

Recommended Posts

I try to make a shadow shader for daylight. For this I use a parallel projection in my shader but I calculate the projection myself without gl_Ortho ( no, I can not use it, let's continue ). Now something is odd with the calculation and even after looking around a lot I could not find a final answer on those questions: 1) Fragment Program: What is the range of gl_FragDepth? [0,1], [-1,1] or [0,2]? 2) Can clipping be disabled? ( so geometry outside the shadow cube can still cast shadows as if they would be sitting at the near clipping plane ) Concerning my tests I would think [-1,1] is the answer to the first question but for the second question I could not find anything. Major problem is that 50% gray squares pop up and out in the shadow map without a reason ( 50% would equal the distance to the ground from the shadow near plane ).

Share this post


Link to post
Share on other sites
You are writing to glFragDepth? The value goes to the depth buffer and the range is 0.0 to 1.0
Anything outside that range should get clipped.

There is a NV extension for disabling clipping
http://www.opengl.org/registry/specs/NV/depth_clamp.txt

Share this post


Link to post
Share on other sites
Okay so the range should be [0,1] then. About the extensions as far as I remember this is nVidia only. I've got though a nice little idea how I can trick the card. Let's test it out :D

EDIT: One more thing. Stumbled across something on the net. They say the depth is in the range [-1,1] before clipping. So would this mean gl_Vertex.z in the vertex program should be in the range [-1,1] but gl_FragDepth in the fragment program [0,1] ?

Share this post


Link to post
Share on other sites
gl_Vertex is the object space vertex and can have any depth value.
The value that gets written to gl_Position will have the z from -1 to 1.
That is then transformed to the 0 to 1 range before depth testing unless if you change it with a call to glDepthRange. In the end, all values must be 0 to 1 flaoting point, which are changed to a integer type (16, 24 or 32 bit) and written to the depth buffer.

Share this post


Link to post
Share on other sites
Okay, I tried doing this but it doesn't work. I tried to trick the graphic card doing something like this:

vertex:
gl_TexCoord[ 0 ] = gl_ModelViewMatrix * vec4( gl_Vertex, 1.0 );
gl_Position = vec4( gl_TexCoord[ 0 ].xy, 0.0, 1.0 );

fragment:
gl_FragDepth = gl_TexCoord[ 0 ].s;

This should give 0 z value during clipping and during fragment it gets back the original depth ( the modelview matrix contained the proper setup to transfer z into 0-1 range ). Nevertheless I've got stripes across the z-buffer and polygon-offset is ignored. I also tried offseting the z value in the fragment program to compensate for the lacking polygon-offset but it is neglected o.O . Doing it the old way with the clipping problem does not produce these artifacts. Why then can we write to gl_FragDepth if writing to it is not allowed?

So now I've got either a clipping problem but right z buffer or no clipping problem but messed up z buffer.

Share this post


Link to post
Share on other sites
For the fragment shader, you are writing the .x values to the depth
gl_FragDepth = gl_TexCoord[0].s;

You probably meant
gl_FragDepth = gl_TexCoord[0].z;

I made a mistake when I said
"The value that gets written to gl_Position will have the z from -1 to 1."
Actually, it can be beyond that depending on your modelview and projection matrix.

Share this post


Link to post
Share on other sites
Sorry, wrong copy paste. Here's the full one:

void main( void ){
vec3 position = ( gl_ModelViewMatrix * vec4( gl_Vertex.xyz, 1.0 ) ).xyz;

gl_TexCoord[ 0 ].s = position.z;

gl_Position = vec4( position.x, position.y, 0.0, 1.0 );
}

void main( void ){
gl_FragDepth = clamp( gl_TexCoord[ 0 ].s, 0.0, 1.0 );
}


the W-Line in the matrix is 0,0,0,1 therefore no division required. I get though stripes. It's strange somehow. If I visualize the shadow map it looks all fine but during rendering I get these awful stripes ( like if the shadow test on the card is failing 50% of the time ).

EDIT: That's how it looks like: image. I noticed that if I use the other approach I get the same stripes back one I do 4-tap PCF instead of 1-tap shadow testing. What I don't get is why. The shadow map in this situation covers a sky light box with dimensions ( 11.7, 27.0, 27.2 ). Therefore the z-buffer has a resolution of 27.2/(2^24) = 1.62e-6 m/px. I read back the depth map and it is correct. The values for each line ( the stripes are coincident with rows in the shadow map ) differ roughly by 0.0002-0.0004 for the given test configuration. I tried reading the shadow map as a texture instead of a shadow map with linear filtering to avoid the stair-case due to pixel resolution yet the same happens. I'm a bit out of ideas on what is going wrong.

EDIT: EDIT:
Something is really wrong here. I went up all the way to give unbelievable 0.5m bias to the depth map and still the stripes prevail. I have no clue what's going on. It can't be a precision problem but what else? It's a simple shadow2D call.

EDIT: EDIT: EDIT:
I measured now the maximal difference in depth values which results in unbelievable 3cm error on a flat surface! What's going on here? The depth buffer is 24-bit in precision. How can 3cm error happen? How is one supposed to make a useful shadow mapper with a hardware yielding 3cm error in depth comparison?

[Edited by - RPTD on July 26, 2008 5:12:05 PM]

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this