# vertex shader output interpolation problem

This topic is 1695 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

I have a question about vertex shader output interpolation. Fox example, when implement shadow map, we need a depth map.  Let's first suppose that we need a non-linear depth map, Z/W. Consider following two shaders, which is the correct way to output a non-linear depth map. I have seen online that someone use the first version, while others use second. But I think only first version is right in math.

// Version 1
void NonLinearDepthVS(in float3 iPos : POSITION, out float4 oPos : SV_Position, out float oDepth : TEXCOORD0)
{
oPos = mul(iPos, mul(World, ViewProj));
oDepth = oPos.z / oPos.w;
}

float4 NonLinearDepthPS(in float oDepth : TEXCOORD0) : SV_Target0
{
return float4(oDepth, 0, 0, 0);
}

// Version 2
void NonLinearDepthVS(in float3 iPos : POSITION, out float4 oPos : SV_Position, out float2 oDepth : TEXCOORD0)
{
oPos = mul(iPos, mul(World, ViewProj));
oDepth = oPos.zw;
}

float4 NonLinearDepthPS(in float2 oDepth : TEXCOORD0) : SV_Target0
{
return float4(oDepth.x / oDepth.y, 0, 0, 0);
}

##### Share on other sites

Unless you're using oDepth somewhere else, it's the same thing. The second version just does the division in the second function.

Edited by Khatharr

##### Share on other sites

Unless you're using oDepth somewhere else, it's the same thing. The second version just does the division in the second function.

Really? I don't think so. The rasterizer will interpolate vertex shader outputs, default is by perspective-correct interpolation. Consider rasterize a triangle, for first version, the final value of a fragment in triangle is a * (Z1/W1) + b * (Z2/W2) + c * (Z3/W3), (a, b, c) is the barycentric coordinate. (Z/W) is values on each triangle vertex. But for second version, the final value is (a*Z1 + b*Z2 + c*Z3) / (a*W1 + b*W2 + c*W3). Obviously, not the same value.

##### Share on other sites

Unless you're using oDepth somewhere else, it's the same thing. The second version just does the division in the second function.

Really? I don't think so. The rasterizer will interpolate vertex shader outputs, default is by perspective-correct interpolation. Consider rasterize a triangle, for first version, the final value of a fragment in triangle is a * (Z1/W1) + b * (Z2/W2) + c * (Z3/W3), (a, b, c) is the barycentric coordinate. (Z/W) is values on each triangle vertex. But for second version, the final value is (a*Z1 + b*Z2 + c*Z3) / (a*W1 + b*W2 + c*W3). Obviously, not the same value.

I would have to check the docs, but my guess is that texcoords are linearly interpolated.  If that's true, then both versions are basically the same, although it'd probably be better to do the division in the vertex shader.

##### Share on other sites

So the output z is divided by w behind the scene, can we then use vertex output position z as input in pixel shader? Should be the same thing.

##### Share on other sites

SV_Position's w coordinate is special -- it's used to implement perspective correct interpolation of ALL vertex outputs.

i.e. oDepth is divided by oPos.w automatically in-between the vertex and pixel shader, during rasterization/interpolation.

So if in the vertex shader you write "oDepth = oPos.z/oPos.w", then in the pixel shader, oDepth will equal oPos.z/oPos.w/oPos.w.

However, none of this is necessary. Just use a depth-stencil target instead of a colour target, and don't do anything special to output depth besides just rasterizing triangles as usual and have the hardware write them to the depth buffer.

Edited by Hodgman

##### Share on other sites

So if in the vertex shader you write "oDepth = oPos.z/oPos.w", then in the pixel shader, oDepth will equal oPos.z/oPos.w/oPos.w.

Now you got me confused. I thought that just output position is affected "behind scene"?

If we saved in vertex shader z/w in oDepth, oDepth should not be affected later on? Or i am wrong?

I am interested in DX9 pipeline, don't know about DX11.

Edited by belfegor

##### Share on other sites

So if in the vertex shader you write "oDepth = oPos.z/oPos.w", then in the pixel shader, oDepth will equal oPos.z/oPos.w/oPos.w.

Now you got me confused. I thought that just output position is affected "behind scene"?

If we saved in vertex shader z/w in oDepth, oDepth should not be affected later on? Or i am wrong?

I am interested in DX9 pipeline, don't know about DX11.

Actually,  perspective correct interpolation means that in order to do linear interpolation, it first need to divide by SV_Position's w. later, it will multiply back before pass the output to pixel shader. So If we saved in vertex shader z/w in oDepth, oDepth is equal to  the interpolated z/w in pixel shader.

##### Share on other sites

OMG.

Forgive my stupidity, can you clear some information.

...it first need to divide by SV_Position's w...

You are talking just about output positions, oDepth is untouched?

...later, it will multiply back before pass the output to pixel shader...

So it divides by w after vertex shader and then just before pixel shader it multiplies by w (restoring old value) ?

##### Share on other sites

SV_Position's w coordinate is special -- it's used to implement perspective correct interpolation of ALL vertex outputs.

i.e. oDepth is divided by oPos.w automatically in-between the vertex and pixel shader, during rasterization/interpolation.

So if in the vertex shader you write "oDepth = oPos.z/oPos.w", then in the pixel shader, oDepth will equal oPos.z/oPos.w/oPos.w.

However, none of this is necessary. Just use a depth-stencil target instead of a colour target, and don't do anything special to output depth besides just rasterizing triangles as usual and have the hardware write them to the depth buffer.

Why would oDepth be divided by oPos.w automatically?  It uses the TEXCOORD0 semantic, not SV_Position.

##### Share on other sites

OMG.

Forgive my stupidity, can you clear some information.

...it first need to divide by SV_Position's w...

You are talking just about output positions, oDepth is untouched?

>

...later, it will multiply back before pass the output to pixel shader...

So it divides by w after vertex shader and then just before pixel shader it multiplies by w (restoring old value) ?

http://www.comp.nus.edu.sg/~lowkl/publications/lowk_persp_interp_techrep.pdf

##### Share on other sites

Why would oDepth be divided by oPos.w automatically?  It uses the TEXCOORD0 semantic, not SV_Position.

The position semantic is special -- it's used by the rasterizer to figure out how all the other values should be interpolated.

i.e. oDepth is divided by oPos.w automatically in-between the vertex and pixel shader, during rasterization/interpolation.
So if in the vertex shader you write "oDepth = oPos.z/oPos.w", then in the pixel shader, oDepth will equal oPos.z/oPos.w/oPos.w.

I had this slightly wrong; what happens in normal usage is that the PS receives:
PS.input.oDepth = interpolate(oPos.z/oPos.w)*interpolate(oPos.w)

if in the vertex shader you write "oDepth = oPos.z/oPos.w", then in the pixel shader you get:
PS.input.oDepth = interpolate(oPos.z/oPos.w/oPos.w)*interpolate(oPos.w)

Say you've got some regular code like this:
VS:
output.pos = mul(...);
output.texcoord = ...
PS:
float color = tex2D( input.texcoord );
The hardware will perform perspective correct interpolation of all your PS-inputs/VS-outputs.
It does this by:
1) At the end of the vertex shader, every interpolant is divided by w.
2) An extra hidden interpolant is created, which holds the value of 1/w.
3) At the start of the pixel shader, every interpolant is divided by the interpolated value of 1/w.

Thanks to the GPU doing this behind the scenes, you get results like in Image A here, otherwise if the GPU didn't do this, you'd get results like in Image B, which looks like PS1 games did...

If you want to disable perspective-correction and see the PS1-esque results for yourself, then at the end of your vertex shader, perform the perspective division yourself (which also sets w to 1, which disables the above 3 steps)
output.position /= output.position.w;//disable hardware perspective correction
If you want to verify that the above 3 steps are what actually happens, then create the hidden "1/w" interpolant yourself and use it in the PS:
VS:
output.texcoord /= output.position.w; // step 1
output.hidden = 1/output.position.w; // step 2
output.position /= output.position.w;//disable hardware perspective correction
PS:
input.texcood /= input.hidden; // step 3
This code should give you the same results as the regular, built-in hardware version... except that doing "output.position /= output.position.w" in the VS can cause the GPU to cause some clipping problems, etc...

However, none of this is necessary. Just use a depth-stencil target instead of a colour target, and don't do anything special to output depth besides just rasterizing triangles as usual and have the hardware write them to the depth buffer.

^This is worth repeating though. If you simply want to output z/w to a texture, then use a depth-stencil target and just let the rasterizer do it all automatically. Edited by Hodgman

##### Share on other sites

Why would oDepth be divided by oPos.w automatically?  It uses the TEXCOORD0 semantic, not SV_Position.

The position semantic is special -- it's used by the rasterizer to figure out how all the other values should be interpolated.

>i.e. oDepth is divided by oPos.w automatically in-between the vertex and pixel shader, during rasterization/interpolation.
So if in the vertex shader you write "oDepth = oPos.z/oPos.w", then in the pixel shader, oDepth will equal oPos.z/oPos.w/oPos.w.

I had this slightly wrong; what happens in normal usage is that the PS receives:
PS.input.oDepth = interpolate(oPos.z/oPos.w)*interpolate(oPos.w)

if in the vertex shader you write "oDepth = oPos.z/oPos.w", then in the pixel shader you get:
PS.input.oDepth = interpolate(oPos.z/oPos.w/oPos.w)*interpolate(oPos.w)

Say you've got some regular code like this:
VS:
output.pos = mul(...);
output.texcoord = ...
PS:
float color = tex2D( input.texcoord );
The hardware will perform perspective correct interpolation of all your PS-inputs/VS-outputs.
It does this by:
1) At the end of the vertex shader, every interpolant is divided by w.
2) An extra hidden interpolant is created, which holds the value of 1/w.
3) At the start of the pixel shader, every interpolant is divided by the interpolated value of 1/w.

Thanks to the GPU doing this behind the scenes, you get results like in Image A here, otherwise if the GPU didn't do this, you'd get results like in Image B, which looks like PS1 games did...

If you want to disable perspective-correction and see the PS1-esque results for yourself, then at the end of your vertex shader, perform the perspective division yourself (which also sets w to 1, which disables the above 3 steps)
output.position /= output.position.w;//disable hardware perspective correction
If you want to verify that the above 3 steps are what actually happens, then create the hidden "1/w" interpolant yourself and use it in the PS:
VS:
output.texcoord /= output.position.w; // step 1
output.hidden = 1/output.position.w; // step 2
output.position /= output.position.w;//disable hardware perspective correction
PS:
input.texcood /= input.hidden; // step 3
This code should give you the same results as the regular, built-in hardware version... except that doing "output.position /= output.position.w" in the VS can cause the GPU to cause some clipping problems, etc...

However, none of this is necessary. Just use a depth-stencil target instead of a colour target, and don't do anything special to output depth besides just rasterizing triangles as usual and have the hardware write them to the depth buffer.

^This is worth repeating though. If you simply want to output z/w to a texture, then use a depth-stencil target and just let the rasterizer do it all automatically.

Thanks to your reply, Actually I know what hardware rasterizer does. I implement a software D3D10 like rasterizer before.

I post this topic just to verify what it the right way to output the non-linear z/w depth if not use depth-stencil target. Shader Version 1 or Version 2 ? Because so many shader online use version 1 while others use version 2.

PS: we can also use type qualifier to control the interpolation method,  See  Interpolation Modifier  in HLSL.

##### Share on other sites

I post this topic just to verify what it the right way to output the non-linear z/w depth if not use depth-stencil target. Shader Version 1 or Version 2 ? Because so many shader online use version 1 while others use version 2.

PS: we can also use type qualifier to control the interpolation method,  See  Interpolation Modifier  in HLSL.

Shader model 1 doesn't even have pixel shaders  If you're using D3D11, then you'll be using SM 2, 4 or 5 (depending if you use the 9, 10 or 11 feature level). Perspective correct interpolation works the same in every shader model, except that in 4/5 you can use the modifiers that you mention.

In SM 2/3, without these modifiers, I guess that if you want to get per-pixel interpolated(z)/interpolated(w), you'd output z*w and w in the vertex shader:

VS output:
pos = float4(x,y,z,w);
o   = float2(z*w, w);

Interpolator performs per pixel:
o.x = interpolate(z*w/w)/interpolate(1/w) == interpolate(z)/interpolate(1/w)
o.y = interpolate(w/w)/interpolate(1/w) == 1/interpolate(1/w)

Ps code:
float depthBuf = o.x / o.y / o.y;
// depthBuf == interpolate(z)/interpolate(1/w) / (1/interpolate(1/w)) / (1/interpolate(1/w))
// depthBuf == interpolate(z)/interpolate(1/w) * interpolate(1/w) * interpolate(1/w)
// depthBuf == interpolate(z) * interpolate(1/w)

This is confusing though, I might have made a mistake

Why not just use a depth buffer?

Edited by Hodgman

##### Share on other sites

I post this topic just to verify what it the right way to output the non-linear z/w depth if not use depth-stencil target. Shader Version 1 or Version 2 ? Because so many shader online use version 1 while others use version 2.

PS: we can also use type qualifier to control the interpolation method,  See  Interpolation Modifier  in HLSL.

Shader model 1 doesn't even have pixel shaders  If you're using D3D11, then you'll be using SM 2, 4 or 5 (depending if you use the 9, 10 or 11 feature level). Perspective correct interpolation works the same in every shader model, except that in 4/5 you can use the modifiers that you mention.

In SM 2/3, without these modifiers, I guess that if you want to get per-pixel interpolated(z)/interpolated(w), you'd output z*w and w in the vertex shader:

VS output:
pos = float4(x,y,z,w);
o   = float2(z*w, w);

Interpolator performs per pixel:
o.x = interpolate(z*w/w)/interpolate(1/w) == interpolate(z)/interpolate(1/w)
o.y = interpolate(w/w)/interpolate(1/w) == 1/interpolate(1/w)

Ps code:
float depthBuf = o.x / o.y / o.y;
// depthBuf == interpolate(z)/interpolate(1/w) / (1/interpolate(1/w)) / (1/interpolate(1/w))
// depthBuf == interpolate(z)/interpolate(1/w) * interpolate(1/w) * interpolate(1/w)
// depthBuf == interpolate(z) * interpolate(1/w)

This is confusing though, I might have made a mistake

Why not just use a depth buffer?

Yeah, instead of output a non-linear depth buffer, I can use the system back depth buffer directly. I just want to know why the version 1 shader is wrong.

I have found that divide w in vertex shader is a wrong way. because w may less than 0. So the version 2 is right.