Sign in to follow this  
PDX_Catalyst

Incorrect Interpolation from Vertex Shader to Pixel Shader

Recommended Posts

I'm getting what looks like poor interpolation in the shading of my terrain. You can see the lines/diamonds in the picture below. I made the terrain pure white to better show them. On a related? note: I'm curious if this same issue might also explain why it seems like the interpolation of the textures is also poor so that the textures blend together poorly in the diagonal directions. Two consecutive vertices with different textures look ok if they're both on the x or the y axis, but vertices diagonal from each other look bad if they have different textures. I'm guessing it's related to the shading problem. [Edited by - PDX_Catalyst on July 27, 2008 2:24:11 PM]

Share this post


Link to post
Share on other sites
By the way, I moved the calculations to the pixel shader and it looked exactly the same. Here's what I'm doing in the vertex shader...

GVS_OUTPUT vs_ground(float4 inPos: POSITION, float3 inNormal: NORMAL, float2 inTexCoords: TEXCOORD0)
{
GVS_OUTPUT Out = (GVS_OUTPUT)0;

Out.Pos = mul(inPos, ViewProjMx);
inNormal = normalize(inNormal);
vector SunVc = mul(SunPos, InvWorldMx);
vector SunDir = normalize(SunVc - inPos);
Out.Color = dot(inNormal, SunDir) * (1 - Ambient) + Ambient;
Out.TexUV = inTexCoords;
return Out;
}

If it's not a problem with the interpolation or this shader code then maybe my normals aren't quite right, but they should be.

Share this post


Link to post
Share on other sites
Yes, just view and projection. The world is always centered on 0, 0 just because of the way the game works, so the world matrix is always equal to Matrix.Identity. Glad you pointed that out because I didn't need to multiply the SunPos by InvWorldMx since it doesn't do anything.

SunPos is in world coordinates.

All that should be ok... the interpolation just seems to be bad. You can see in the picture that all four vertices surrounding a pair of triangles are darker than the triangles, which makes no sense since the fill should be no darker or lighter than the darkest or lightest vertex. I'm stumped.

Share this post


Link to post
Share on other sites
Okay. I thought maybe you were mixing apples and oranges by dotting the unmodified normal with whatever the SunVC would've been. [smile]

Quote:
maybe my normals aren't quite right, but they should be.


Are you using 3 vertices per face, or shared vertices for your mesh?

If 3 vertices per face, are you sure that the normals for vertices at the same position, but on different faces, are identical?

Also, assuming that, in your world, the sun is far enough away from your vertices, you might try using a sun direction, rather than SunPos-inPos. It would save you 1 calculation and would avoid subtracting a very small number (inPos) from (I assume) a very large one (SunPos), which, possibly, could be a problem.

Share this post


Link to post
Share on other sites
You're right, the SunPos - inPos was unnecessary. I just took out the subtraction and it works the same.

I'm using shared vertices.

Here's how I'm getting the vertice normals:

First I store the normals of each triangle ...

plane = new Plane(groundVertices[x + y * SIZE_V].Position, groundVertices[x + (y + 1) * SIZE_V].Position, groundVertices[x + 1 + y * SIZE_V].Position);

triNorm[x + y * SIZE_Q, 0] = plane.Normal;

... then I go through and average the normals for all 6 triangles connected to each vertice ...

groundVertices[x + y * SIZE_V].Normal =
(
triNorm[x - 1 + (y - 1) * SIZE_Q, 1] +
triNorm[x + y * SIZE_Q, 0] +
triNorm[x + (y - 1) * SIZE_Q, 0] +
triNorm[x + (y - 1) * SIZE_Q, 1] +
triNorm[x - 1 + y * SIZE_Q, 0] +
triNorm[x - 1 + y * SIZE_Q, 1]
)
/ 6f;

... so hopefully that's done properly.

Share this post


Link to post
Share on other sites
For an unweighted average, your calc is correct. A long time ago, I tried:

((x-1,y-1),0) +
((x,y-1),0)*0.5 +
((x,y-1),1)*0.5 +
((x,y),0) +
((x-1,y-1),0)*0.5 +
((x-1,y-1),1)*0.5

Then divide by 4 and normalize.

That seemed to help.

Might be worth a try.

Share this post


Link to post
Share on other sites
Quote:
just normalize the normal for every vertex..

The code he is posting is the process to determine the normal for each vertex. The vertex normal doesn't exist until this process is done. [smile]

Share this post


Link to post
Share on other sites
That's funny, using the logic that the ".5" triangles are "less attached", I tried exactly that before and it didn't seem to do anything about the patterns. I wrote the normal code by thinking out what needed to happen, not by learning it somewhere, so I wasn't absolutely sure I'd done it right. Glad to know I was on the right track. I'll try tweaking those numbers more extremely and see how that works out, because my feeling was also that something was weighted incorrectly. HOWEVER, regardless of what the normals surrounding a triangle are it still doesn't make sense to me that the triangle could be lighter than all three vertices. I mean, logically, if you interpolate between .5, .8, and .3 how could you ever get .9 or .2?

So we can at least agree this shading appearance is not normal, right? It SHOULD look smooth, right? I spent a bunch of time looking for examples of someone else's terrain (or any mesh really) with the same pattern in the shading, but didn't find anything to show either way.

I appreciate the help, BTW =)

Share this post


Link to post
Share on other sites
Well, I played with the numbers, but it only changed the shade amounts, not the pattern, and I realized that logically that's what I should expect. Changing the normals will change the amount of shading contributed by each vertex, but the amount of shading at each vertex point isn't the problem. The problem is the way the three shade amounts are interpolating across the triangle. This also seems to be an issue with the interpolation of other numbers being passed from the vertex shader. For instance when I pass a float4 that holds texture information it seems to interpolate incorrectly as well... though I haven't looked into that one yet.

I really wish I could get in there and write that interpolation formula myself.

Any other thoughts?

Share this post


Link to post
Share on other sites
Ok, so it's just the way bilinear sampling is. I created a small image that had one pixel per vertice, each colored the correct shade amount. I put that image into Photoshop and blew it up using bilinear resampling. It had the same patterns in it. Bicubic also had that pattern. Bicubic Smoother looked great.

So... I guess the only way to get rid of those patterns is to do the sampling myself in the pixel shader (from a terrain/shade map) and use a formula similar to what Photoshop uses for Bicubic Smoother.

Sounds expensive. =(

Share this post


Link to post
Share on other sites
Make sure you aren't normalizing your normals in the vertex shader.
Otherwise you are getting gouraud shading which results in a diamond shading pattern.

This link explains it nicely (search for normalize).
http://wiki.truevision3d.com/hlsl_intrinsics

I had this problem a while back. Most noticable with specular highlights.

Share this post


Link to post
Share on other sites
I tried that but I still got the same lines. I figured out with photoshop that the lines are just a product of the linear interpolation (as described in my previous reply). The only solution I've come up with so far is making a map of the shading at each vertex and then taking a sort of blurred sample of it. That's only really works because it's for a grid terrain. Wouldn't help if I were shading a model or something.

I also have three texture strengths in the map (shade in alpha). The blurred sampling makes for some nice texture transitions too.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this