I'm having some problems with depth peeling. I have previously implemented this before successfully on older hardware, but I cant get my new version to work. I used to simply call into tex2dproj with projective transformed vertexes, but the profile our engine uses with cg for w/e reason no longer supports tex2dproj, so I have to scale the texture vertexes into the required range myself. This seems to be working, but I am having a lot of trouble with the comparison, the results are very low precision. I almost never seem to get any rejection of fragments, unless I use a basis, then the results become very inaccurate.
Here is the code that scales the vertex positions into the range [0,1]
float4x4 ScaleTexture = float4x4(float4(.5,0,0,0),float4(0,.5,0,0),float4(0,0,1,0),float4(0,0,0,1));float4 finalposition = mul(ProjectionMatrix, pos);OUT.Position = finalposition;float4(((finalposition).xyz/(finalposition.w))+float3(1,1,0),1));projectedtex.y = 1 - projectedtex.y;OUT.TexCoord3 = projectedtex;
Pixel Shader
MRT_OUT OUT; color.a = (IN.TexCoord3.z); if(firstlayer > 0) { if( color.a <= tex2D(Texture2, (IN.TexCoord3.xy) ).r)//Depth peeling section { discard; } OUT.rt2 = color.a; } else { OUT.rt2 = float4(color.a,color.a,color.a,color.a); }
I have checked in PIX and the R32 Texture being passed in as depth appears to be correct. Unfortunately I cant seem to debug the pixel shader(pix complains when I try and do this). Anything seem obviously wrong?
Regards,
Jesse