GLSL normal mapping strange behaviour

Started by
9 comments, last by Styves 10 years, 2 months ago

Hi,

I was trying to add a normal map effect to a shader tutorial I have found here but it's strangely flickering O.O

The vertex shader code:

UPDATE 2:

http://www.gamedev.net/topic/654758-glsl-normal-mapping-strange-behaviour/?view=findpost&p=5141727

UPDATE 1:

http://www.gamedev.net/topic/654758-glsl-normal-mapping-strange-behaviour/?view=findpost&p=5141553


#version 330


in vec3 inPosition;
in vec3 vertNormal;
in vec2 vertTexCoord;
in vec4 vertNormalMapping;

out vec3 surfacePos;
out vec3 fragNormal;
out vec2 fragTexCoord;
out vec4 fragNormalMapping;

uniform mat4 modelViewProjectionMatrix;
uniform mat4 camera;

void main(){	
    fragTexCoord = vertTexCoord;
    fragNormal = vertNormal;
    surfacePos = vec3(modelViewProjectionMatrix * vec4(inPosition, 1));
    fragNormalMapping = vertNormalMapping;



    gl_Position  = camera * modelViewProjectionMatrix * vec4(inPosition, 1.0);

	
	
}

The fragment shader


#version 330

precision highp float;


uniform vec3 cameraPosition;
uniform mat4 modelViewProjectionMatrix;
uniform sampler2D tex;
uniform sampler2D normalMap; 



uniform float materialShininess;
uniform vec3 materialSpecularColor;

uniform struct Light {
   vec3 position;
   vec3 intensities; //a.k.a the color of the light
   float attenuation;
   float ambientCoefficient;
} light;


in vec3 fragNormal;
in vec3 surfacePos;
in vec2 fragTexCoord;
in vec4 fragNormalMapping;

out vec4 finalColor;

void main() {

	
	
	 vec3 normalMap;
	 
	if(fragNormalMapping[0] == 1.0f){	//has a normal map?			
		normalMap =  texture(normalMap, fragTexCoord).xyz * 2.0 - 1.0;
		normalMap.g = 1.0-normalMap.g;

		normalMap = normalize(normalMap);
	}else{ //No normal map texture so just normal calculation
		normalMap = normalize(transpose(inverse(mat3(modelViewProjectionMatrix))) * (fragNormal));
	}
	


   
   
    vec4 surfaceColor = texture(tex, fragTexCoord);
    vec3 surfaceToLight = normalize(light.position - surfacePos);
    vec3 surfaceToCamera = normalize(cameraPosition - surfacePos);
    
    //ambient
    vec3 ambient = light.ambientCoefficient * surfaceColor.rgb * light.intensities;

    //diffuse0.0
    float diffuseCoefficient =max( dot(normalMap, surfaceToLight),0.0);
    vec3 diffuse = diffuseCoefficient * surfaceColor.rgb * light.intensities;
    
    //specular
    float specularCoefficient = 0.0;

    if(diffuseCoefficient > 0.0){		
		specularCoefficient = pow(max(0.0, dot(surfaceToCamera, reflect(-surfaceToLight, normalMap))), materialShininess);       
	}
    vec3 specular = specularCoefficient * materialSpecularColor * light.intensities;
    
    //attenuation
    float distanceToLight = length(light.position - surfacePos);
    float attenuation = 1.0 / (1.0 + light.attenuation * pow(distanceToLight, 2));

    //linear color (color before gamma correction)	 
	vec3 linearColor = ambient + attenuation*(diffuse + specular);
   
    
    //final color (after gamma correction)
    vec3 gamma = vec3(1.0/2.2);

    finalColor = vec4(pow(linearColor, gamma), surfaceColor.a);
}

For better testing I'm using a plane with a one color texture and a bump texture

this is the result

ibahVfbbiKuHRO.jpg

The light is following the camera and I can see that the normal map is calculated correctly. But as you can see there is something wrong with the "shadows" and I really don't know what it is.

If I load objects without a bump map the result with the same shader is perfect.

ib01goUaO1QMeA.jpg

thank you everyone for the help =D

Advertisement

Don't calculate surfacePos using the (wv-)projection matrix; transform only by the world-transform part.

Don't calculate surfacePos using the (wv-)projection matrix; transform only by the world-transform part.

seems like nothing changed.. oh, yes, the normal map changed but the lines are still there :|

Later I will post the other code. Thank you


if(fragNormalMapping[0] == 1.0f){	//has a normal map?			
		normalMap =  texture(normalMap, fragTexCoord).xyz * 2.0 - 1.0;
		normalMap.g = 1.0-normalMap.g;

		normalMap = normalize(normalMap);
	}else{ //No normal map texture so just normal calculation
		normalMap = normalize(transpose(inverse(mat3(modelViewProjectionMatrix))) * (fragNormal));
	}

this seems suspicious to me. In case of no normal map, you tranform object space normal to projection space, but, in case of texture normal, you just sample from texture and do not transform it to projection space. You need to create tangent space matrix, and apply tangent*world*view*projection transformation to sampled normal from texture, unless you use object space normals in your normal map.

Thanks everyone. I edited the code, here is the new version

Vertex shader


#version 330


in vec3 inPosition;
in vec3 vertNormal;
in vec2 vertTexCoord;
in vec4 vertNormalMapping;

out vec3 fragVert;
out vec3 fragNormal;
out vec2 fragTexCoord;
out vec4 fragNormalMapping;
out mat3 TBNMatrix;

uniform mat4 modelViewProjectionMatrix;
uniform mat4 camera;

void main(){	

	vec3 tangent; 
	vec3 binormal; 

	vec3 c1 = cross( vertNormal, vec3(0.0, 0.0, 1.0) ); 
	vec3 c2 = cross( vertNormal, vec3(0.0, 1.0, 0.0) ); 

	if( length(c1)>length(c2) )
	{
		tangent = c1;	
	}
	else
	{
		tangent = c2;	
	}

	tangent = normalize(tangent);

	binormal = cross(vertNormal, tangent); 
	binormal = normalize(binormal);

	mat3 normalMatrix = transpose(inverse(mat3(camera * modelViewProjectionMatrix )));
	vec3 n = normalize(normalMatrix * vertNormal);
	vec3 t = normalize(normalMatrix * tangent.xyz);
	vec3 b = normalize(normalMatrix * binormal.xyz);
	TBNMatrix = mat3(t, b, n);

	

	fragTexCoord = vertTexCoord;
    fragNormal = vertNormal;
	fragVert = inPosition;
	fragNormalMapping = vertNormalMapping;


	gl_Position  = camera * modelViewProjectionMatrix * vec4(inPosition, 1.0);

	
	
}

Fragment shader


#version 330

precision highp float;


uniform vec3 cameraPosition;
uniform mat4 modelViewProjectionMatrix;
uniform mat4 camera;
uniform sampler2D tex;
uniform sampler2D normalMap; 



uniform float materialShininess;
uniform vec3 materialSpecularColor;

uniform struct Light {
   vec3 position;
   vec3 intensities; //a.k.a the color of the light
   float attenuation;
   float ambientCoefficient;
} light;


in vec3 fragNormal;
in vec3 fragVert;
in vec2 fragTexCoord;
in vec4 fragNormalMapping;
in mat3 TBNMatrix;

out vec4 finalColor;

void main() {

	vec3 surfacePos = vec3(modelViewProjectionMatrix * vec4(fragVert, 1));
    vec4 surfaceColor = texture(tex, fragTexCoord);
    vec3 surfaceToLight = TBNMatrix * (light.position - surfacePos) ;
    vec3 surfaceToCamera = TBNMatrix * (cameraPosition - surfacePos);



	vec3 normal = normalize(texture(normalMap, fragTexCoord).xyz * 2.0 - 1.0);
	

	//ambient
	vec3 ambient = light.ambientCoefficient * surfaceColor.rgb * light.intensities;
	
	//diffuse
    float diffuseCoefficient = max(0.0, dot(normal, surfaceToLight));
    vec3 diffuse = diffuseCoefficient * surfaceColor.rgb * light.intensities;

	//specular
    float specularCoefficient = 0.0;
    if(diffuseCoefficient > 0.0)
        specularCoefficient = pow(max(0.0, dot(surfaceToCamera, reflect(-surfaceToLight, normal))), materialShininess);
    vec3 specular = specularCoefficient * materialSpecularColor * light.intensities;
    
    //attenuation
    float distanceToLight = length(light.position - surfacePos);
    float attenuation = 1.0 / (1.0 + light.attenuation * pow(distanceToLight, 2));

    //linear color (color before gamma correction)
    vec3 linearColor = ambient + attenuation*(diffuse + specular);
    
    //final color (after gamma correction)
    vec3 gamma = vec3(1.0/2.2);
    finalColor = vec4(pow(linearColor, gamma), surfaceColor.a);
	
}

There is no more flickering and I see something that someone can say "mhm... it seems like this should be a normal map" but the light is... seems random O.O

When I rotate the light rotate with me in another direction creating some very strange darker sides somewhere O.O

You see this line? yeah. Maybe tomorrow I see where I'm wrong

i5J3tdtjFF4gU.jpg


if(fragNormalMapping[0] == 1.0f){
        normal = normalize(texture(heightMap, fragTexCoord).xyz * 2.0 - 1.0);
    }else{
        normal = normalize(texture(heightMap, fragTexCoord).xyz * 2.0 - 1.0);
    }

Errr I think you have a typo

You are sampling the height map in both cases, shouldn't one be the normal map?


if(fragNormalMapping[0] == 1.0f){
        normal = normalize(texture(heightMap, fragTexCoord).xyz * 2.0 - 1.0);
    }else{
        normal = normalize(texture(heightMap, fragTexCoord).xyz * 2.0 - 1.0);
    }

Errr I think you have a typo

You are sampling the height map in both cases, shouldn't one be the normal map?

woops sorry, I edited. I did this for compile the shader without changing the c++ code xD I had to use somewhere fragNormalMapping variable for compile it =D

Update of the code

I have added a correct TBN matrix... or I hope.


vert shader


#version 330


in vec3 inPosition;
in vec3 vertNormal;
in vec2 vertTexCoord;
in vec4 vertNormalMapping;

out vec3 fragVert;
out vec3 fragNormal;
out vec2 fragTexCoord;
out vec4 fragNormalMapping;
out mat3 TBNMatrix;
out mat3 normalMatrix;

uniform mat4 transform;
uniform mat4 camera;
uniform mat4 proj;

void main(){	

	vec3 tangent; 
	
	
	vec3 c1 = cross( vertNormal, vec3(0.0, 0.0, 1.0) ); 
	vec3 c2 = cross( vertNormal, vec3(0.0, 1.0, 0.0) ); 
	vec3 b;
	if( length(c1)>length(c2) )
	{
		tangent = c1;	
	}
	else
	{
		tangent = c2;	
	}

	tangent = normalize(tangent);
	
	

	mat4 mv = transform  * camera;//modelview
	mat4 mvp = proj * camera * transform;//modelviewprojection
	mat4 mvi = transpose(inverse(mv));//modelview inverse (=gl_NormalMatrix)

	mat3 normalMatrix = mat3(mvi);
	vec3 n = normalize( ( normalMatrix *  vertNormal )  );
	vec3 t = normalize( ( normalMatrix *  tangent.xyz ) );
	b = cross(vertNormal, tangent); 
	b = normalize(normalMatrix * b);

	TBNMatrix = mat3(t, b, n);
	TBNMatrix = transpose(TBNMatrix);
	

	fragTexCoord = vertTexCoord;
    fragNormal = vertNormal;
	fragVert = inPosition;
	fragNormalMapping = vertNormalMapping;



	gl_Position  = mvp * vec4(inPosition, 1.0);

	
	
}


fragment shader


#version 330

precision highp float;


uniform vec3 cameraPosition;
uniform mat4 transform;
uniform mat4 camera;
uniform sampler2D tex;
uniform sampler2D normalMap; 



uniform float materialShininess;
uniform vec3 materialSpecularColor;

uniform struct Light {
   vec3 position;
   vec3 intensities; //a.k.a the color of the light
   float attenuation;
   float ambientCoefficient;
} light;


in vec3 fragNormal;
in vec3 fragVert;
in vec2 fragTexCoord;
in vec4 fragNormalMapping;
in mat3 TBNMatrix;
in mat3 normalMatrix;

out vec4 finalColor;

void main() {

	vec3 surfacePos = vec3(transform  * vec4(fragVert,1.0));
    vec4 surfaceColor = texture(tex, fragTexCoord);
    vec3 surfaceToLight = normalize( light.position - surfacePos)  ;
    vec3 surfaceToCamera = normalize( cameraPosition - surfacePos) ;



	vec3 normal;
	if(fragNormalMapping[0] == 1.0f){
		normal = normalize(texture(normalMap, fragTexCoord).xyz * 2.0 - 1.0);
	}else{
		normal = normalize(texture(normalMap, fragTexCoord).xyz * 2.0 - 1.0);
	}

	//ambient
	vec3 ambient = light.ambientCoefficient * surfaceColor.rgb * light.intensities;
	
	//diffuse
    float diffuseCoefficient = max(0.0, dot(normal, surfaceToLight));
    vec3 diffuse = diffuseCoefficient * surfaceColor.rgb * light.intensities;

	//specular
    float specularCoefficient = 0.0;
    if(diffuseCoefficient > 0.0)
        specularCoefficient = pow(max(0.0, dot(surfaceToCamera, reflect(-surfaceToLight, normal))), materialShininess);
    vec3 specular = specularCoefficient * materialSpecularColor * light.intensities;
    
    //attenuation
    float distanceToLight = length(light.position - surfacePos);
    float attenuation = 1.0 / (1.0 + light.attenuation * pow(distanceToLight, 2));

    //linear color (color before gamma correction)
    vec3 linearColor = ambient + attenuation*(diffuse + specular);
    
    //final color (after gamma correction)
    vec3 gamma = vec3(1.0/2.2);
    finalColor = vec4(pow(linearColor, gamma), surfaceColor.a);
	
} 

The result is still wrong :-(

ior1DN7SXURAI.jpg

mhm... O.O

First thing is first: you're not even using the TBN! :D You need to take the TBN and multiply it to your normal map, otherwise what's the point? tongue.png

A side note: I don't trust your TBN calculations as they don't actually use the triangle data, but some arbitrary directions. This isn't a trustworthy approach and even if you use it, I don't expect proper results from it.

Instead, try this in your fragment shader (pulled from my engine, you're welcome happy.png):


mat3 ComputeCotangentFrame( vec3 vNormal, vec3 vPosition, vec2 vTexCoord )
{
	vec3 ddxPos = dFdx(vPosition);
	vec3 ddyPos = dFdy(vPosition);
	vec2 ddxUV = dFdx(vTexCoord);
	vec2 ddyUV = dFdy(vTexCoord);

	vec3 vCrossVec1 = cross( ddyPos, vNormal );
	vec3 vCrossVec2 = cross( vNormal, ddxPos );
	vec3 vTangent = vCrossVec1 * ddxUV.x + vCrossVec2 * ddyUV.x;
	vec3 vBinormal = vCrossVec1 * ddxUV.y + vCrossVec2 * ddyUV.y;

	float fDotT = dot( vTangent, vTangent );
	float fDotB = dot( vBinormal, vBinormal );
	float fInvMax = 1.0 / sqrt(max(fDotT, fDotB));

	return mat3( vTangent * fInvMax, vBinormal * fInvMax, vNormal );
}

Inputs are vertex normal, world position and texture coordinates.

Next, why do you apply a transform matrix to the vertex position in the fragment shader? This is most definitely better done in the vertex shader and passed in.

And finally, what's the point of this:


vec3 normal;
	if(fragNormalMapping[0] == 1.0f){
		normal = normalize(texture(normalMap, fragTexCoord).xyz * 2.0 - 1.0);
	}else{
		normal = normalize(texture(normalMap, fragTexCoord).xyz * 2.0 - 1.0);
	}

You're doing the same thing twice regardless of the branch, so why bother?

Once you get everything working, you can look into replacing the Cotangent stuff I supplied with standard tangent/binormal calculations on the CPU and pass them in as vertex attributes. Unless you don't mind the performance overhead of computing it all on the fly per-pixel (you'll gain some memory savings by not storing them ;D), or you're doing all shading/materials on the GPU in a procedural deferred pass like I am... which I highly doubt anyone is doing. tongue.png

Whoa, thank you.

Sorry for the "fragNormalMapping", it is initialized and I have to use it somewhere for build the shader D= I need to change the c++ code, I know :-( I will do it when everything is working =D

Your code is very usefull, now I can make some testing. I have written the code (with the help of google and this guy www.terathon.com/code/tangent.html ) for make the calculatio in CPU, but for now let see if it's working with your code.

I have only a question that maybe... maybe it's a little stupid. The sampler2d normalMap variable is just a RGB as it is from the normal map texture. Do I have to normalize it to -1 and +1 via CPU or the code I have should work?

This topic is closed to new replies.

Advertisement