# Lighting artifacts

This topic is 670 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

Hello, i found problem with lighting in my terrain generator. Where is the problem? Any ideas?

Where is the problem?
Where is the problem?

##### Share on other sites

Looks like you haven't calculated smoothed normals for your terrain. The normal at each vertex should be an average of all faces/triangles to which it's connected. To me it looks like the lighting is being done according to the triangle normal.

##### Share on other sites

But i calculated normals:

float3 tangent = normalize(float3(2.0f*gWorldCellSpace, rightY - leftY, 0.0f));
float3 bitan   = normalize(float3(0.0f, bottomY - topY, -2.0f*gWorldCellSpace));
float3 normalW = cross(tangent, bitan);



And its probably ok.

##### Share on other sites

That code looks like it's calculating the normal of the triangle as I said above. If you have the same normal used across the entire triangle then your terrain is going to appear as a series of connected triangles rather than a smooth surface.

##### Share on other sites

ok, so tesselation should help ? or how can i improve normals? I detect something strange, problem exist  only in high terrains like moutains ;/

Normals calculate:

float2 leftTex   = pin.Tex + float2(-gTexelCellSpaceU, 0.0f);
float2 rightTex  = pin.Tex + float2(gTexelCellSpaceU, 0.0f);
float2 bottomTex = pin.Tex + float2(0.0f, gTexelCellSpaceV);
float2 topTex    = pin.Tex + float2(0.0f, -gTexelCellSpaceV);

float leftY   = gHeightMap.SampleLevel( samHeightmap, leftTex, 0 ).r;
float rightY  = gHeightMap.SampleLevel( samHeightmap, rightTex, 0 ).r;
float bottomY = gHeightMap.SampleLevel( samHeightmap, bottomTex, 0 ).r;
float topY    = gHeightMap.SampleLevel( samHeightmap, topTex, 0 ).r;

float3 tangent = normalize(float3(2.0f*gWorldCellSpace, rightY - leftY, 0.0f));
float3 bitan   = normalize(float3(0.0f, bottomY - topY, -2.0f*gWorldCellSpace));
float3 normalW = cross(tangent, bitan);


Edited by widmowyfox

##### Share on other sites

Where is this code located?  It looks like you're calculating the normal for a single cell.  If you're calculating that per cell, then passing that normal to all 4 vertices in the cell, then the entire cell will appear flat (like you're screenshot).  As Adam pointed out, you need to calculate the average normal between cells, for for every vertex you need to average it with the neighboring cells.

11--12--13--14
| A | B | C |
21--22--23--24
| D | E | F |
31--32--33--34
| G | H | I |
41--33--43--44


I assume that currently you have each cell contain it's own 4 vertices (so A has 11/12/21/22), and each vertex has the normal of that cell.  To smooth out the normals 22 needs to average the normals from cells A/B/D/E since it's shared with all 4 cells.  Even if you separate all cells into their own 4 unique vertices, to smooth out the normals you still need to treat them all as shared by the neighbors.

Edited by xycsoscyx

##### Share on other sites

ok, but why this problem exist only in hight terrain generation? If it is as you say it should be everywhere. Can you give me a sample how i should solve it?

Edited by widmowyfox

##### Share on other sites

Howe to solve it really depends on the answer to my previous question.  Where are you using the code that you posted?  Is it in a geometry shader where you're generating the vertex for each cell, is it in a vertex shader, is it in a pixel shader, other?  The simple answer is exactly what we've said, you need to smooth out your normals per vertex by averaging them with all neighboring cells.

Vertex22.normal = (FaceA.normal + FaceB.normal + FaceD.normal + FaceE.Normal) / 4


Why you only see it on height terrain generation also requires an answer.  If you're generating the cells one at a time, 4 vertex per cell, all in a geometry shader, then each cell will be independent and have it's own flat normal.  The code you posted, regardless of how/where you're using it, only generates a single normal for an entire cell, that means that entire cell will only face in a single direction and won't be smooth.

Edited by xycsoscyx

##### Share on other sites

Check your index buffer for vertex index order. It looks like the two triangles, do not form correct face, thus causing the wrong values to be interpolated from vertex to pixel shader. Why I think this is a case i,s because even your UVs appear to be off.

##### Share on other sites

Is it possible that when the area is folded it reduces the number of triangles to the area and this causes high accuracy normal ?

Below complete shader that explain where is previously written code.

#include "LightHelper.fx"

cbuffer cbPerFrame
{
DirectionalLight gDirLights[3];
float3 gEyePosW;

float  gFogStart;
float  gFogRange;
float4 gFogColor;

float gMinDist;
float gMaxDist;

float gMinTess;
float gMaxTess;

float gTexelCellSpaceU;
float gTexelCellSpaceV;
float gWorldCellSpace;
float2 gTexScale = 500.0f;

float4 gWorldFrustumPlanes[6];
};

cbuffer cbPerObject
{

float4x4 gViewProj;
Material gMaterial;

};

Texture2DArray gLayerMapArray;
Texture2D gBlendMap;
Texture2D gHeightMap;

SamplerState samLinear
{
Filter = MIN_MAG_MIP_LINEAR;

};

SamplerState samHeightmap
{
Filter = MIN_MAG_LINEAR_MIP_POINT;

};

{
Filter   = COMPARISON_MIN_MAG_LINEAR_MIP_POINT;
BorderColor = float4(0.0f, 0.0f, 0.0f, 0.0f);

ComparisonFunc = LESS;
};

struct VertexIn
{
float3 PosL     : POSITION;
float2 Tex      : TEXCOORD0;
float2 BoundsY  : TEXCOORD1;
};

struct VertexOut
{
float3 PosW     : POSITION;
float2 Tex      : TEXCOORD0;
float2 BoundsY  : TEXCOORD1;

};

VertexOut VS(VertexIn vin)
{
VertexOut vout;

vout.PosW = vin.PosL;

vout.PosW.y = gHeightMap.SampleLevel( samHeightmap, vin.Tex, 0 ).r;

vout.Tex      = vin.Tex;
vout.BoundsY  = vin.BoundsY;

return vout;
}

float CalcTessFactor(float3 p)
{
float d = distance(p, gEyePosW);

//float d = max( abs(p.x-gEyePosW.x), abs(p.z-gEyePosW.z) );

float s = saturate( (d - gMinDist) / (gMaxDist - gMinDist) );

return pow(2, (lerp(gMaxTess, gMinTess, s)) );
}

bool AabbBehindPlaneTest(float3 center, float3 extents, float4 plane)
{
float3 n = abs(plane.xyz);

float r = dot(extents, n);

float s = dot( float4(center, 1.0f), plane );

return (s + r) < 0.0f;
}

bool AabbOutsideFrustumTest(float3 center, float3 extents, float4 frustumPlanes[6])
{
for(int i = 0; i < 6; ++i)
{

if( AabbBehindPlaneTest(center, extents, frustumPlanes[i]) )
{
return true;
}
}

return false;
}

struct PatchTess
{
float EdgeTess[4]   : SV_TessFactor;
float InsideTess[2] : SV_InsideTessFactor;
};

PatchTess ConstantHS(InputPatch<VertexOut, 4> patch, uint patchID : SV_PrimitiveID)
{
PatchTess pt;

float minY = patch[0].BoundsY.x;
float maxY = patch[0].BoundsY.y;

float3 vMin = float3(patch[2].PosW.x, minY, patch[2].PosW.z);
float3 vMax = float3(patch[1].PosW.x, maxY, patch[1].PosW.z);

float3 boxCenter  = 0.5f*(vMin + vMax);
float3 boxExtents = 0.5f*(vMax - vMin);
if( 1==0 )
{
pt.EdgeTess[0] = 0.0f;
pt.EdgeTess[1] = 0.0f;
pt.EdgeTess[2] = 0.0f;
pt.EdgeTess[3] = 0.0f;

pt.InsideTess[0] = 0.0f;
pt.InsideTess[1] = 0.0f;

return pt;
}

else
{

float3 e0 = 0.5f*(patch[0].PosW + patch[2].PosW);
float3 e1 = 0.5f*(patch[0].PosW + patch[1].PosW);
float3 e2 = 0.5f*(patch[1].PosW + patch[3].PosW);
float3 e3 = 0.5f*(patch[2].PosW + patch[3].PosW);
float3  c = 0.25f*(patch[0].PosW + patch[1].PosW + patch[2].PosW + patch[3].PosW);

pt.EdgeTess[0] = CalcTessFactor(e0);
pt.EdgeTess[1] = CalcTessFactor(e1);
pt.EdgeTess[2] = CalcTessFactor(e2);
pt.EdgeTess[3] = CalcTessFactor(e3);

pt.InsideTess[0] = CalcTessFactor(c);
pt.InsideTess[1] = pt.InsideTess[0];

return pt;
}
}

struct HullOut
{
float3 PosW     : POSITION;
float2 Tex      : TEXCOORD0;

};

[partitioning("fractional_even")]
[outputtopology("triangle_cw")]
[outputcontrolpoints(4)]
[patchconstantfunc("ConstantHS")]
[maxtessfactor(64.0f)]
HullOut HS(InputPatch<VertexOut, 4> p,
uint i : SV_OutputControlPointID,
uint patchId : SV_PrimitiveID)
{
HullOut hout;

hout.PosW     = p[i].PosW;
hout.Tex      = p[i].Tex;

return hout;
}

struct DomainOut
{
float4 PosH     : SV_POSITION;
float3 PosW     : POSITION;
float2 Tex      : TEXCOORD0;
float2 TiledTex : TEXCOORD1;

};

DomainOut DS(PatchTess patchTess,
float2 uv : SV_DomainLocation,
{
DomainOut dout;

dout.PosW = lerp(
uv.y);

dout.Tex = lerp(
uv.y);

dout.TiledTex = dout.Tex*gTexScale;

dout.PosW.y = gHeightMap.SampleLevel( samHeightmap, dout.Tex, 0 ).r;

dout.PosH    = mul(float4(dout.PosW, 1.0f), gViewProj);

return dout;
}

float4 PS(DomainOut pin,
uniform int gLightCount,
uniform bool gFogEnabled) : SV_Target
{

float2 leftTex   = pin.Tex + float2(-gTexelCellSpaceU, 0.0f);
float2 rightTex  = pin.Tex + float2(gTexelCellSpaceU, 0.0f);
float2 bottomTex = pin.Tex + float2(0.0f, gTexelCellSpaceV);
float2 topTex    = pin.Tex + float2(0.0f, -gTexelCellSpaceV);

float leftY   = gHeightMap.SampleLevel( samHeightmap, leftTex, 0 ).r;
float rightY  = gHeightMap.SampleLevel( samHeightmap, rightTex, 0 ).r;
float bottomY = gHeightMap.SampleLevel( samHeightmap, bottomTex, 0 ).r;
float topY    = gHeightMap.SampleLevel( samHeightmap, topTex, 0 ).r;

float3 tangent = normalize(float3(2.0f*gWorldCellSpace, rightY - leftY, 0.0f));
float3 bitan   = normalize(float3(0.0f, bottomY - topY, -2.0f*gWorldCellSpace));
float3 normalW = cross(tangent, bitan);

float3 toEye = gEyePosW - pin.PosW;

float distToEye = length(toEye);

toEye /= distToEye;

float4 c0 = gLayerMapArray.Sample( samLinear, float3(pin.TiledTex, 0.0f) );
float4 c1 = gLayerMapArray.Sample( samLinear, float3(pin.TiledTex, 1.0f) );
float4 c2 = gLayerMapArray.Sample( samLinear, float3(pin.TiledTex, 2.0f) );
float4 c3 = gLayerMapArray.Sample( samLinear, float3(pin.TiledTex, 3.0f) );
float4 c4 = gLayerMapArray.Sample( samLinear, float3(pin.TiledTex, 4.0f) );

float4 t  = gBlendMap.Sample( samLinear, pin.Tex );

float4 texColor = c0;
texColor = lerp(texColor, c1, t.r);
texColor = lerp(texColor, c2, t.g);
texColor = lerp(texColor, c3, t.b);
texColor = lerp(texColor, c4, t.a);

float4 litColor = texColor;
if( gLightCount > 0  )
{
// Rozpocznij sumowanie od zera.
float4 ambient = float4(0.0f, 0.0f, 0.0f, 0.0f);
float4 diffuse = float4(0.0f, 0.0f, 0.0f, 0.0f);
float4 spec    = float4(0.0f, 0.0f, 0.0f, 0.0f);

float3 shadow = float3(1.0f, 1.0f, 1.0f);

[unroll]
for(int i = 0; i < 1; ++i)
{
float4 A, D, S;
ComputeDirectionalLight(gMaterial, gDirLights[i], normalW, toEye,
A, D, S);

ambient += A;

}

litColor = texColor*(ambient + diffuse) + spec;
}

if( gFogEnabled )
{
float fogLerp = saturate( (distToEye - gFogStart) / gFogRange );

// Zmieszaj kolor mg?y i o?wietlony kolor.
litColor = lerp(litColor, gFogColor, fogLerp);
}

return litColor;
}


Edited by widmowyfox

##### Share on other sites

It still seems like you're using a single block from the height map to calculate the normal for every pixel in the quad.  This means that every pixel will wind up with the same normal since they're all reading from the same block in the height map.

Slidexx01 may be right though because it does look like the texture coordinates per quad are rotated somehow.

EDIT: Looking closer, it looks like the textures might be lining up properly, so that may not be it (though worth checking).  Can you calculate the normals in the HS function instead?  Generate them higher up, per vertex, then tessellate them like everything else, then let them be interpolated across the faces normally.

Edited by xycsoscyx

##### Share on other sites

Thanks all, i solved problem by using better tessellation.

Edited by widmowyfox

##### Share on other sites

I back to the topic again, problem still exist  I just did not check well enough. None of the solutions do not work. I add shader file:

#include "LightHelper.fx"

cbuffer cbPerFrame
{
DirectionalLight gDirLights[3];
float3 gEyePosW;

float  gFogStart;
float  gFogRange;
float4 gFogColor;

float gMinDist;
float gMaxDist;

float gMinTess;
float gMaxTess;

float gTexelCellSpaceU;
float gTexelCellSpaceV;
float gWorldCellSpace;
float2 gTexScale = 50.0f;

float4 gWorldFrustumPlanes[6];
};

cbuffer cbPerObject
{

float4x4 gViewProj;
Material gMaterial;
};

Texture2DArray gLayerMapArray;
Texture2D gBlendMap;
Texture2D gHeightMap;

SamplerState samLinear
{
Filter = MIN_MAG_MIP_LINEAR;

};

SamplerState samHeightmap
{
Filter = MIN_MAG_LINEAR_MIP_POINT;

};

struct VertexIn
{
float3 PosL     : POSITION;
float2 Tex      : TEXCOORD0;
float2 BoundsY  : TEXCOORD1;
};

struct VertexOut
{
float3 PosW     : POSITION;
float2 Tex      : TEXCOORD0;
float2 BoundsY  : TEXCOORD1;
};

VertexOut VS(VertexIn vin)
{
VertexOut vout;

vout.PosW = vin.PosL;

vout.PosW.y = gHeightMap.SampleLevel( samHeightmap, vin.Tex, 0 ).r;

vout.Tex      = vin.Tex;
vout.BoundsY  = vin.BoundsY;

return vout;
}

float CalcTessFactor(float3 p)
{
float d = distance(p, gEyePosW);

// odleg?o?? obliczana na p?aszczy?nie xz (co pozwala zobaczy? poziomy szczegó?ów z lotu ptaka).
//float d = max( abs(p.x-gEyePosW.x), abs(p.z-gEyePosW.z) );

float s = saturate( (d - gMinDist) / (gMaxDist - gMinDist) );

return pow(2, (lerp(gMaxTess, gMinTess, s)) );
}

bool AabbBehindPlaneTest(float3 center, float3 extents, float4 plane)
{
float3 n = abs(plane.xyz);

float r = dot(extents, n);

float s = dot( float4(center, 1.0f), plane );

return (s + r) < 0.0f;
}

bool AabbOutsideFrustumTest(float3 center, float3 extents, float4 frustumPlanes[6])
{
for(int i = 0; i < 6; ++i)
{
// Je?eli prostopad?o?cian znajduje si? w ca?o?ci za jedn? z p?aszczyzn ostros?upa,
// le?y na zewn?trz ostros?upa.
if( AabbBehindPlaneTest(center, extents, frustumPlanes[i]) )
{
return true;
}
}

return false;
}

struct PatchTess
{
float EdgeTess[4]   : SV_TessFactor;
float InsideTess[2] : SV_InsideTessFactor;
};

PatchTess ConstantHS(InputPatch<VertexOut, 4> patch, uint patchID : SV_PrimitiveID)
{
PatchTess pt;

float minY = patch[0].BoundsY.x;
float maxY = patch[0].BoundsY.y;

float3 vMin = float3(patch[2].PosW.x, minY, patch[2].PosW.z);
float3 vMax = float3(patch[1].PosW.x, maxY, patch[1].PosW.z);

float3 boxCenter  = 0.5f*(vMin + vMax);
float3 boxExtents = 0.5f*(vMax - vMin);
if( AabbOutsideFrustumTest(boxCenter, boxExtents, gWorldFrustumPlanes) )
{
pt.EdgeTess[0] = 0.0f;
pt.EdgeTess[1] = 0.0f;
pt.EdgeTess[2] = 0.0f;
pt.EdgeTess[3] = 0.0f;

pt.InsideTess[0] = 0.0f;
pt.InsideTess[1] = 0.0f;

return pt;
}

else
{

float3 e0 = 0.5f*(patch[0].PosW + patch[2].PosW);
float3 e1 = 0.5f*(patch[0].PosW + patch[1].PosW);
float3 e2 = 0.5f*(patch[1].PosW + patch[3].PosW);
float3 e3 = 0.5f*(patch[2].PosW + patch[3].PosW);
float3  c = 0.25f*(patch[0].PosW + patch[1].PosW + patch[2].PosW + patch[3].PosW);

pt.EdgeTess[0] = CalcTessFactor(e0);
pt.EdgeTess[1] = CalcTessFactor(e1);
pt.EdgeTess[2] = CalcTessFactor(e2);
pt.EdgeTess[3] = CalcTessFactor(e3);

pt.InsideTess[0] = CalcTessFactor(c);
pt.InsideTess[1] = pt.InsideTess[0];

return pt;
}
}

struct HullOut
{
float3 PosW     : POSITION;
float2 Tex      : TEXCOORD0;
};

[partitioning("fractional_even")]
[outputtopology("triangle_cw")]
[outputcontrolpoints(4)]
[patchconstantfunc("ConstantHS")]
[maxtessfactor(64.0f)]
HullOut HS(InputPatch<VertexOut, 4> p,
uint i : SV_OutputControlPointID,
uint patchId : SV_PrimitiveID)
{
HullOut hout;

hout.PosW     = p[i].PosW;
hout.Tex      = p[i].Tex;

return hout;
}

struct DomainOut
{
float4 PosH     : SV_POSITION;
float3 PosW     : POSITION;
float2 Tex      : TEXCOORD0;
float2 TiledTex : TEXCOORD1;
};

DomainOut DS(PatchTess patchTess,
float2 uv : SV_DomainLocation,
{
DomainOut dout;

dout.PosW = lerp(
uv.y);

dout.Tex = lerp(
uv.y);

dout.TiledTex = dout.Tex*gTexScale;

dout.PosW.y = gHeightMap.SampleLevel( samHeightmap, dout.Tex, 0 ).r;

dout.PosH    = mul(float4(dout.PosW, 1.0f), gViewProj);

return dout;
}

float4 PS(DomainOut pin,
uniform int gLightCount,
uniform bool gFogEnabled) : SV_Target
{

float2 leftTex   = pin.Tex + float2(-gTexelCellSpaceU, 0.0f);
float2 rightTex  = pin.Tex + float2(gTexelCellSpaceU, 0.0f);
float2 bottomTex = pin.Tex + float2(0.0f, gTexelCellSpaceV);
float2 topTex    = pin.Tex + float2(0.0f, -gTexelCellSpaceV);

float leftY   = gHeightMap.SampleLevel( samHeightmap, leftTex, 0 ).r;
float rightY  = gHeightMap.SampleLevel( samHeightmap, rightTex, 0 ).r;
float bottomY = gHeightMap.SampleLevel( samHeightmap, bottomTex, 0 ).r;
float topY    = gHeightMap.SampleLevel( samHeightmap, topTex, 0 ).r;

float3 tangent = normalize(float3(2.0f*gWorldCellSpace, rightY - leftY, 0.0f));
float3 bitan   = normalize(float3(0.0f, bottomY - topY, -2.0f*gWorldCellSpace));
float3 normalW = cross(tangent, bitan);

float3 toEye = gEyePosW - pin.PosW;

float distToEye = length(toEye);

toEye /= distToEye;

float4 c0 = gLayerMapArray.Sample( samLinear, float3(pin.TiledTex, 0.0f) );
float4 c1 = gLayerMapArray.Sample( samLinear, float3(pin.TiledTex, 1.0f) );
float4 c2 = gLayerMapArray.Sample( samLinear, float3(pin.TiledTex, 2.0f) );
float4 c3 = gLayerMapArray.Sample( samLinear, float3(pin.TiledTex, 3.0f) );
float4 c4 = gLayerMapArray.Sample( samLinear, float3(pin.TiledTex, 4.0f) );

float4 t  = gBlendMap.Sample( samLinear, pin.Tex );

float4 texColor = c0;
texColor = lerp(texColor, c1, t.r);
texColor = lerp(texColor, c2, t.g);
texColor = lerp(texColor, c3, t.b);
texColor = lerp(texColor, c4, t.a);

float4 litColor = texColor;
if( gLightCount > 0  )
{

float4 ambient = float4(0.0f, 0.0f, 0.0f, 0.0f);
float4 diffuse = float4(0.0f, 0.0f, 0.0f, 0.0f);
float4 spec    = float4(0.0f, 0.0f, 0.0f, 0.0f);

[unroll]
for(int i = 0; i < gLightCount; ++i)
{
float4 A, D, S;
ComputeDirectionalLight(gMaterial, gDirLights[i], normalW, toEye,
A, D, S);

ambient += A;
diffuse += D;
spec    += S;
}

litColor = texColor*(ambient + diffuse) + spec;
}

if( gFogEnabled )
{
float fogLerp = saturate( (distToEye - gFogStart) / gFogRange );

litColor = lerp(litColor, gFogColor, fogLerp);
}

return litColor;
}

technique11 Light1
{
pass P0
{
}
}

technique11 Light2
{
pass P0
{
}
}

technique11 Light3
{
pass P0
{
}
}

technique11 Light1Fog
{
pass P0
{
}
}

technique11 Light2Fog
{
pass P0
{
}
}

technique11 Light3Fog
{
pass P0
{
}
}



Light compute function:

void ComputeDirectionalLight(Material mat, DirectionalLight L,
float3 normal, float3 toEye,
out float4 ambient,
out float4 diffuse,
out float4 spec)
{

ambient = float4(0.0f, 0.0f, 0.0f, 0.0f);
diffuse = float4(0.0f, 0.0f, 0.0f, 0.0f);
spec    = float4(0.0f, 0.0f, 0.0f, 0.0f);

float3 lightVec = -L.Direction;

ambient = mat.Ambient * L.Ambient;

float diffuseFactor = dot(lightVec, normal);

[flatten]
if( diffuseFactor > 0.0f )
{
float3 v         = reflect(-lightVec, normal);
float specFactor = pow(max(dot(v, toEye), 0.0f), mat.Specular.w);

diffuse = diffuseFactor * mat.Diffuse * L.Diffuse;
spec    = specFactor * mat.Specular * L.Specular;
}
}

Another picture:

Edited by widmowyfox

##### Share on other sites

This topic is 670 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Create an account

Register a new account

• ### Forum Statistics

• Total Topics
628633
• Total Posts
2983956

• 10
• 18
• 20
• 13
• 9