Sign in to follow this  
Mange86

GLSL tangent space problems

Recommended Posts

Hi.Im getting quite awkward results while using my normal mapping shader.

I am using one directional light. All my lighting seem to be correct when using my other shaders, so i do believe the problem lies with my TBN matrix.

Picture 1 shows the result of the normal mapping. I get quite hard edges on the duck, Picture 2, for instance but other then that i think the result is ok. However, i am not 100% sure about the ducks texture coordinates and that normal map was randomly taken from the internet. The other two models should however behave.

Picture 3 and 4 shows the results of just lighting in tangent space using the vertex normals specified with the models. It looks horrible. The house gets lit from both directions, the duck looks like a chessboard etc...
[attachment=5756:Skärmavbild 2011-10-13 kl. 10.53.02.png] [attachment=5757:Skärmavbild 2011-10-13 kl. 10.53.52.png][attachment=5758:Skärmavbild 2011-10-13 kl. 10.54.50.png] [attachment=5759:Skärmavbild 2011-10-13 kl. 10.55.00.png]

This is the tangent calculation code:
[code]
typedef struct V_T_N { float v[3], vt[2], vn[3], vtang[4];} VertTexNormTang;
typedef struct INDICE { unsigned int i[3];} Indice;

void ModelLoader::generateTangents(GeometryPtr temp) {
std::vector indices = temp->getIndices();
std::vector vertTexNormTangs = temp->getVertTexNorms();
for (unsigned int i = 0; i < indices.size(); i++) {
float *pos1 = vertTexNormTangs[indices[i].i[0]].v;
float *pos2 = vertTexNormTangs[indices[i].i[1]].v;
float *pos3 = vertTexNormTangs[indices[i].i[2]].v;
float *tex1 = vertTexNormTangs[indices[i].i[0]].vt;
float *tex2 = vertTexNormTangs[indices[i].i[1]].vt;
float *tex3 = vertTexNormTangs[indices[i].i[2]].vt;
float *norm1 = vertTexNormTangs[indices[i].i[0]].vn;
float *norm2 = vertTexNormTangs[indices[i].i[1]].vn;
float *norm3 = vertTexNormTangs[indices[i].i[2]].vn;

Vector4f edge1(pos2[0] - pos1[0], pos2[1] - pos1[1], pos2[2] - pos1[2]);
Vector4f edge2(pos3[0] - pos1[0], pos3[1] - pos1[1], pos3[2] - pos1[2]);
edge1.normalize();
edge2.normalize();

Vector4f texEdge1(tex2[0] - tex1[0], tex2[1] - tex1[1], 0);
Vector4f texEdge2(tex3[0] - tex1[0], tex3[1] - tex1[1], 0);
texEdge1.normalize();
texEdge2.normalize();

Vector4f t, b, n(norm1[0]+norm2[0]+norm3[0], norm1[1]+norm2[1]+norm3[1], norm1[2]+norm2[2]+norm3[2]); n.normalize();
float det = (texEdge1.getX() * texEdge2.getY()) - (texEdge1.getY() * texEdge2.getX());
if (fabs(det) < 0.00001) {
t = Vector4f(1.0f, 0.0f, 0.0f);
b = Vector4f(0.0f, 1.0f, 0.0f);
} else {
det = 1.0f/det;
t.setX((texEdge2.getY() * edge1.getX() - texEdge1.getY() * edge2.getX()) * det);
t.setY((texEdge2.getY() * edge1.getY() - texEdge1.getY() * edge2.getY()) * det);
t.setZ((texEdge2.getY() * edge1.getZ() - texEdge1.getY() * edge2.getZ()) * det);

b.setX((-texEdge2.getX() * edge1.getX() + texEdge1.getX() * edge2.getX()) * det);
b.setY((-texEdge2.getX() * edge1.getY() + texEdge1.getX() * edge2.getY()) * det);
b.setZ((-texEdge2.getX() * edge1.getZ() + texEdge1.getX() * edge2.getZ()) * det);

t.normalize();
b.normalize();
}
Vector4f bitangent = n.cross(t);
float handness = (bitangent.dot(b) < 0.0f) ? -1.0f : 1.0f;
for (int j = 0; j < 3; j++) {
vertTexNormTangs[indices[i].i[j]].vtang[0] = t.getX();
vertTexNormTangs[indices[i].i[j]].vtang[1] = t.getY();
vertTexNormTangs[indices[i].i[j]].vtang[2] = t.getZ();
vertTexNormTangs[indices[i].i[j]].vtang[3] = handness; }
}

temp->setVertTexNorms(vertTexNormTangs);
}
[/code]

Here is the vertex shader:
[code]
attribute vec4 tangentIn;
varying vec3 normal, tangent, bitangent, temp;
varying vec4 position;

void main() {
temp = gl_Normal;
normal = normalize(gl_NormalMatrix * gl_Normal);
tangent = normalize(gl_NormalMatrix * vec3(tangentIn));
bitangent = cross(normal, tangent) * tangentIn.w;
position = (gl_ModelViewMatrix * gl_Vertex);

gl_TexCoord[0] = gl_MultiTexCoord0;
gl_Position = ftransform();
}
[/code]

Fragment:
[code]
uniform sampler2D diffuseTexture;
uniform sampler2D normalTexture;

void calculateLighting(in mat3 base, in vec4 position, in vec3 n, in float shininess, inout vec4 ambient, inout vec4 diffuse, inout vec4 specular);
void applyTextureDiffuse(inout vec4 color);

varying vec3 normal, tangent, bitangent, temp;
varying vec4 position;

void main (void) {
vec3 n = normalize(normal);
vec3 t = normalize(tangent);
vec3 b = normalize(bitangent);

mat3 base = mat3(t.x, b.x, n.x, t.y, b.y, n.y, t.z, b.z, n.z);

//n = normalize(temp);
n = normalize(vec3(texture2D(normalTexture, gl_TexCoord[0].xy)) * 2.0 - 1.0);

vec4 ambient = vec4(0.0);
vec4 diffuse = vec4(0.0);
vec4 specular = vec4(0.0);

calculateLighting(base, position, n, gl_FrontMaterial.shininess, ambient, diffuse, specular);

vec4 color = gl_FrontLightModelProduct.sceneColor + (ambient * gl_FrontMaterial.ambient) + (diffuse * gl_FrontMaterial.diffuse) + (specular * gl_FrontMaterial.specular);

//applyTextureDiffuse(color);

color = clamp(color, 0.0, 1.0);
color.w = 1.0;
gl_FragColor = color;
}
[/code]

Where i switch between [code]n = normalize(temp);[/code] and [code]n = normalize(vec3(texture2D(normalTexture, gl_TexCoord[0].xy)) * 2.0 - 1.0);[/code] to switch between the two results above, the later one giving the more correct lighting in this case.

The lighting code(calculate lighting iterates thru all the enabled lights and calls this one):
[code]
void directionalLight(in int i, in mat3 base, in vec3 n, in float shininess, inout vec4 ambient, inout vec4 diffuse, inout vec4 specular) {
vec3 lightDir = (base * vec3(gl_LightSource[i].position));
vec3 halfVector = (base * vec3(gl_LightSource[i].halfVector));
vec3 l = normalize(lightDir); float nDotl = dot(n, l);
if (nDotl > 0.0) {
vec3 h = normalize(halfVector);
float pf = pow(max(dot(n,h), 0.0), shininess);
diffuse += (gl_LightSource[i].diffuse * nDotl);
specular += (gl_LightSource[i].specular * pf);
}
ambient += (gl_LightSource[i].ambient);}
[/code]

Ive been at this for a few days now searching around and i can't seem to get my head around the problem. The lighting should look just as normal when using the specified vertex normal instead of the normal map values right? What am i doing wrong in my calculations?

Edit: i saw i forgot to change the name of my method when i added tangents, setVertTexNorms in the tangent calculation actually sets everything on my Geometry to what has use been calculated (included are tangents).
Edit2: The edit ate my line breaks...

Share this post


Link to post
Share on other sites
[quote name='Mange86' timestamp='1318498227' post='4872146']
Ive been at this for a few days now searching around and i can't seem to get my head around the problem. The lighting should look just as normal when using the specified vertex normal instead of the normal map values right?

[/quote] Yes, that's right. Your lighting looks off. Try rendering using only per fragment lighting without the normal map. It looks like you're doing flat shading. Generally, that would mean that your vertex normals are not smooth.

Share this post


Link to post
Share on other sites
The vertex normal is screwed up ... I guess this from the color of the roof in the non-normal mapped version. Presumably the normal map normal would be screwed up too, the images should not look the same images created with the correct pre-calculated vertex normals.

Also the shaders use a specific model (from my out of date memory) ... i.e. open gl and d3d used the same matrices but they were transposed and so the definition of how to calculate a vector in tangent space changes.

Share this post


Link to post
Share on other sites
Just one comment on the duck: It looks like the models normals are screwed up. I had a similar issue some time before, which came from calculating the normals in runtime without using adjancey information (don't know the correct word form, I mean the facing next to the current face) so the normals for each face where taken as it is eithout smoothing them over the model, thus causing such a "checkerboard"-effect. Are you sure the normals of that duck are ok?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this