Hi so I've been trying to calculate normals and lighting for randomly generated terrain. Largely the process has been sucessful however I have notice one odd effect. If you look at the picture below you will see that I get an odd checker effect on the terrain on poorly lit portions.
[attachment=8868:Untitled.jpg]
The edges appear dark but the center is bright. I believe this to be an issue with calculation of the surface normals. To test this I have disabled lighting and output the normals as color in my fragment shader. I see the same effect.
Here's the code I use to calculate the surface normals.

Vert normalizeVert(Vert old_vert) {
Vert new_vert;
float old_length;
old_length = sqrt(pow(old_vert.xcorr, 2.0f) + pow(old_vert.ycorr, 2.0f) + pow(old_vert.zcorr, 2.0f));
new_vert.xcorr = old_vert.xcorr / old_length;
new_vert.ycorr = old_vert.ycorr / old_length;
new_vert.zcorr = old_vert.zcorr / old_length;
return new_vert;
}
;
// Calculate normals for terrain.
void NSheet::calculateNormals(void) {
// First create an object to store all the surface normals
unsigned int num_surfaceNormals = pow(sheet_size - 1, 2) * 2;
Vert* surfaceNormals = new Vert[num_surfaceNormals];
Face* faces = amesh->faces;
// temporary vertices used in calculation
Vert vert1, vert2, vert3, temp_vert;
//retrieve pointer to indices
unsigned int* indices = faces->indices;
// retrieve pointer to mesh data
float* verts = faces->verts;
// compute all surface normals
for (unsigned int i = 0; i < num_surfaceNormals; i++) {
vert1.xcorr = verts[indices[i * 3 + 0] * 4 + 0];
vert1.ycorr = verts[indices[i * 3 + 0] * 4 + 1];
vert1.zcorr = verts[indices[i * 3 + 0] * 4 + 2];
vert2.xcorr = verts[indices[i * 3 + 1] * 4 + 0];
vert2.ycorr = verts[indices[i * 3 + 1] * 4 + 1];
vert2.zcorr = verts[indices[i * 3 + 1] * 4 + 2];
vert3.xcorr = verts[indices[i * 3 + 2] * 4 + 0];
vert3.ycorr = verts[indices[i * 3 + 2] * 4 + 1];
vert3.zcorr = verts[indices[i * 3 + 2] * 4 + 2];
// Calculate surface normal. % overloaded to produce cross product
surfaceNormals = (vert2 - vert1) % (vert3 - vert1);
// normalize surface normal
surfaceNormals = normalizeVert(surfaceNormals);
}
// create an array to hold vertex normal data
faces->num_normals = pow(sheet_size, 2) * 3;
amesh->num_normals = faces->num_normals;
faces->normals = new float[faces->num_normals];
// This variable represents that distance in the normal array between y units in the array of surface normals
unsigned int y_step = 2 * (sheet_size - 1);
// calculate the normal values for each vertex
for (unsigned int i = 0; i < sheet_size; i++)
for (unsigned int j = 0; j < sheet_size; j++) {
// the current location in the surface normal array
unsigned int displacement = i * y_step + j * 2;
// the current location in the vertex normal array
unsigned int vertex_displacement = (i * sheet_size + j) * 3;
// blank the temp vertex
temp_vert.xcorr = 0.0f;
temp_vert.ycorr = 0.0f;
temp_vert.zcorr = 0.0f;
// average the vertices for the surrounding polygons
if (i > 0 && j < sheet_size - 1) { // if has square at top right
temp_vert = temp_vert + (surfaceNormals[displacement - y_step] * 2);
}
if (i > 0 && j > 0) { // if has square at top left
temp_vert = temp_vert + surfaceNormals[displacement - y_step - 2] + surfaceNormals[displacement - y_step - 1];
}
if (i < sheet_size - 1 && j < sheet_size - 1) { // if has square at bottom right
temp_vert = temp_vert + surfaceNormals[displacement] + surfaceNormals[displacement + 1];
}
if (i < sheet_size - 1 && j > 0) { // if square at bottom left
temp_vert = temp_vert + (surfaceNormals[displacement - 1] * 2);
}
// normalize the temporary vertex
temp_vert = normalizeVert(temp_vert);
// set the normal data
faces->normals[vertex_displacement] = temp_vert.xcorr;
faces->normals[vertex_displacement + 1] = temp_vert.ycorr;
faces->normals[vertex_displacement + 2] = temp_vert.zcorr;
}
amesh->has_normals = true; // set to true automatically once normals are calculated
// Free memory
delete[] surfaceNormals;
}


It uses that standard method that I have seen where you calculate a surface normal for each polygon and then average those for the verts. My best guess is that my error is here somewhere but I can't figure out where...

Anyways thanks for taking the time to read this post.

It's because you are averaging the vertex normals for quads, rather than triangles. Triangles by their nature are flat, but your quads are actually formed of two triangles, so they have a crease in the middle. Linear interpolation of normals on the GPU doesn't take the crease into account, so you get bright spots...

The simplest solution is to convert your quads to pairs of triangles, and calculate normals per-triangle int he first pass (rather than per-quad). Even better solution is to switch to per-pixel lighting after that.

Hmmm I don't like to disagree with someone helpful enough to post a response, but unless there is an error in my code, I believe I am calculating it per triangle. At least that was my intention when I wrote the code...

[color=#000000]pow[color=#666600]([color=#000000]sheet_size [color=#666600]- [color=#006666]1[color=#666600], [color=#006666]2[color=#666600]) [color=#666600]* [color=#006666]2 gives the number of triangles not quads. i.e. number of quads * 2.

Then I use the index values (which represent triangles) to set the vertex values prior to computing the cross-product...
So I should be calculating it per triangle if my code is correct.

Unless there's an error you can find which accidently results in it actually being per quad...

Maybe I gave that impression since I have comments refering to "squares at top left" etc... That's just used to consider the position of the vertex so I don't go out of bounds on the edges. What I sum up are the normals of the two triangles. At least that's what I intended.

Please let me know if there is some code accidently making me calculate or sum up quad normals instead...

I will check out per-pixel lighting however

[I will check out per-pixel lighting however

Oh turns out I already am doing per-pixel lighting. I pass normals to the vertex shader which interpolates them when passing to the fragment shader. Then I calculating the lighting per-pixel in the fragment shader using the interpolated normals. I wasn't aware that there was any other way to do it...

What really confuses me is how all 3 corners of each triangle are shaded dark and two of the edges are dark as expected, but the one edge (the one across the middle of the quad) is shaded light.
Strange enough if I make it display only wireframe all 3 edges are dark are expected
Could it have something to do with how opengl iterpolates between the points? Maybe there's an opengl state I need to tweek?

By default a perspective correct linear interpolation is used - and that is what you want. Just do not forget to re-normalize the normals per fragment as interpolation obviously breaks it.

Last i had problems with normals i made a trivial geometry shade to send my geometry to, might be worth sharing (in case you have not used a geometry shader before):

// vertex shader
#version 330
precision highp float;
layout(location=0) in vec3 vxPos;
layout(location=1) in vec3 vxNormal;
layout(std140) uniform Transform { mat4 transform; } u; // some way to get your transform
out Position { vec4 position; } gs;
const float scale = 1.0; // for adjusting drawn normal length
void main() {
gl_Position = u.transform * vec4(vxPos, 1.0);
gs.position = u.transform * vec4(vxPos + vxNormal * scale, 1.0);
}
// geometry shader
#version 330
precision highp float;
layout(points) in;
in Position { vec4 position; } gs[1];
layout(line_strip, max_vertices=2) out;
void main() {
gl_Position = gl_in[0].gl_Position; EmitVertex();
gl_Position = gs[0].position; EmitVertex();
}
// fragment shader
#version 330
precision highp float;
layout(location=0) out vec4 fbColor;
void main() {
fbColor = vec4(1.0);
}


Just send another copy of your usual world geometry to this shader program to be drawn as GL_POINTS. Helps to ensure the normals are the way you assume they are and spot errors if not. Easier to see where the vertices are and hence judge what is interpolated with what etc.

Thanks [color=#284B72]tanzanite7 I'll implement those shaders next.

In the meantime I have rewritten and greately simplified my code. I decided that keeping track of the surface normals was unecessary and just complicated the code, due to having to ensure that they're later assigned to the appropriate vertex.

Instead whenever I calculate a surface normal I immediately add it to the normal of the contributing vectors. This ensures that each vertex normal always gets the correct surface normal contribution. All the normal computation and assingment happens within the second small loop. The first and last loop just load the vertex data and offload the normal data respectively.


// This struct gives a vertex a surface normal
struct LinkedVert : Vert{
Vert normal;
};

// Calculate normals for terrain.
void NSheet::calculateNormals(void) {

unsigned int num_surfaceNormals = pow(sheet_size - 1, 2) * 2;

// create a temporary array to hold vertices temporarily
LinkedVert* verts = new LinkedVert[amesh->num_verts/4];

// pointer to mesh data
Face* mesh_data = amesh->faces;

// First load vertex data and set normal to 0,0,0
for (unsigned int i = 0; i < amesh->num_verts/4; i++){
verts.xcorr = mesh_data->verts[i*4];
verts.ycorr = mesh_data->verts[i*4+1];
verts.zcorr = mesh_data->verts[i*4+2];
verts.normal.xcorr = 0.0f;
verts.normal.ycorr = 0.0f;
verts.normal.zcorr = 0.0f;
}

//retrieve pointer to indices
unsigned int* indices = mesh_data->indices;

// compute surface normals and add to normal of contributing vertex
for (unsigned int i = 0; i < num_surfaceNormals; i++) {
Vert surfaceNormal;
// index values
unsigned int index1 = indices[i*3];
unsigned int index2 = indices[i*3 + 1];
unsigned int index3 = indices[i*3 + 2];
// Calculate surface normal. % overloaded to produce cross product
surfaceNormal = (verts[index2] - verts[index1]) % (verts[index3] - verts[index1]);
// normalize surface normal
surfaceNormal = normalizeVert(surfaceNormal);
// add normal to normal of whichever vert was used to calculate it
verts[index1].normal += surfaceNormal;
verts[index2].normal += surfaceNormal*2.0f; // normal contributes twice to second vertex
verts[index3].normal += surfaceNormal;
}

// create an array to store normal data for later buffering
mesh_data->num_normals = pow(sheet_size, 2) * 3;
amesh->num_normals = mesh_data->num_normals;
mesh_data->normals = new float[mesh_data->num_normals];

// Normalize and transfer all vertex normals to normal array
for (unsigned int i = 0; i < amesh->num_verts/4; i++){
verts.normal = normalizeVert(verts.normal);
mesh_data->normals[i*3] = verts.normal.xcorr;
mesh_data->normals[i*3+1] = verts.normal.ycorr;
mesh_data->normals[i*3+2] = verts.normal.zcorr;
}

amesh->has_normals = true; // set to true automatically once normals are calculated
// Free memory
delete[] verts;
}


I've also included an illustration of how I index the vertices. The second vertex index in each triangle corresponds to the top left and bottom right corners of the containing quad. Thus whenever I add normal data I add it twice for these vertices to properly weight the triangles. In red are the triangles I add per vertex.
[attachment=8906:indexorder.jpg]

At this point since I'm getting the same result with this new method I suspect that my problem lies elsewhere. I'll try that geometry shader and examine more closely my vector math. I overloaded quite a few operators and may have made a small mistake

Last i had problems with normals i made a trivial geometry shade to send my geometry to, might be worth sharing (in case you have not used a geometry shader before):

Just send another copy of your usual world geometry to this shader program to be drawn as GL_POINTS. Helps to ensure the normals are the way you assume they are and spot errors if not. Easier to see where the vertices are and hence judge what is interpolated with what etc.

Hey thanks for the geometry shader. I was able to implement it and display the normals. They appear fine to me from what I can tell
[attachment=8907:normal display.png]

Maybe the problem is in my shader code?

// Vertex shader
#version 330
uniform mat4 projectionMatrix;
uniform mat4 viewMatrix;
uniform mat4 modelMatrix;
uniform mat4 normalMatrix;
layout(location = 0) in vec4 in_position;
layout(location = 1) in vec4 in_colour;
layout(location = 2) in vec3 in_normal;
out vec4 pass_colour;
smooth out vec3 vNormal;
void main()
{
gl_Position = projectionMatrix * viewMatrix * modelMatrix * in_position;
vec4 ad_normal = normalMatrix*vec4(in_normal, 0.0);
vNormal = ad_normal.xyz;
pass_colour = in_colour;
}


// Fragment Shader
#version 330
in vec4 pass_colour;
smooth in vec3 vNormal;
out vec4 out_colour;
struct SimpleDirectionalLight
{
vec3 vColor;
vec3 vDirection;
float fAmbientIntensity;
};
uniform SimpleDirectionalLight sunLight;
void main()
{
float fDiffuseIntensity = max(0.0, dot(normalize(vNormal), -sunLight.vDirection));
out_colour = pass_colour*vec4(sunLight.vColor*min(sunLight.fAmbientIntensity+fDiffuseIntensity,1.0), 1.0);
}