Problems with GLSL Bumpmapping Shader (with Lumina example)

This topic is 3229 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

Recommended Posts

Hi, i am trying to write an GLSL Bumpmapping shader. To try it out, i use Lumina. I know there is an exmaple coming with Lumina, but i would like to have one, that i understand. The way, the code looks right for me, when i run it, i see my object with applied bumpmapping, but when the object is rotated in space, the bumpmapping does not update. I think i somehow lose the orientation of the object in my vertex shader. (=> See screenshots) My approach is as found often in tutorials: Vertex Shader: 1) calculate the TBN-matrix for the current vertex 2) transform the light direction into the tangential space with the TBN-matrix 3) pass this light direction to the fragment shader Fragment shader: 4) calculate the dot product of the light direction and the current normalmaps normal 5) use this value to lighten the pixel My code is the following: Vertex shader:
attribute vec3 tangent;
vec3 lightDirGlobal = vec3(1.0, 0.0, 0.0);
varying vec3 lightDirTangential;

void main(void){
gl_Position = ftransform();
gl_TexCoord[0] = gl_MultiTexCoord0;

vec3 N = gl_Normal;
vec3 T = tangent;
vec3 B = cross(T, N);

mat3 TBNMatrix = mat3(T, B, N);

lightDirTangential = TBNMatrix * lightDirGlobal;
}


uniform sampler2D texture;
uniform sampler2D normalmap;

varying vec3 lightDirTangential;

void main(void){
vec4 texture = texture2D(texture, vec2(gl_TexCoord[0]));
vec3 bumpnormal = texture2D(normalmap, vec2(gl_TexCoord[0])).xyz;

float strength = max(dot(bumpnormal, lightDirTangential), 0.0);

gl_FragColor = vec4(strength * texture.rgb, 1.0);
}


Has anybody an idea, what could be the problem? Why is the orientation of my object lost in transformation? Get the Lumina file and my two test textures BR Janosch EDIT: please ignore my bad english :)

Share on other sites
What's happening here:
	mat3 TBNMatrix = mat3(T, B, N);	lightDirTangential = TBNMatrix * lightDirGlobal;

is:
		|1.0|	 [T,B,N]*|0.0|=T		|0.0|

so you are always sending the -untransformed- tangent vector to your fragment shader!

Share on other sites
Quote:
 My approach is as found often in tutorials:Vertex Shader:1) calculate the TBN-matrix for the current vertex2) transform the light direction into the tangential space with the TBN-matrix3) pass this light direction to the fragment shaderFragment shader:4) calculate the dot product of the light direction and the current normalmaps normal5) use this value to lighten the pixel

1) because you transformed the vertex into the camera's coordinates system, you have to do the same with the TBN matrix.(assuming the modelview matrix is a combination of only rotations and translations you can use gl_NormalMatrix).
2)the TBN matrix transforms a vector from tangent space. So you actually need it's inverse [sad]. This is why it's better to send the TBN matrix as varying to the fragment shader in order to use it to transform the normal retrived from the normal-map.(EDIT: Well, because the TBNMatrix is orthogonal, it's inverse is equal to it's transpose [embarrass])

So here is how it should be done (code not tested)
attribute vec3 tangent;varying mat3 TBNMatrix;void main(void){	gl_Position = ftransform();	gl_TexCoord[0] = gl_MultiTexCoord0;	vec3 N = gl_Normal;	vec3 T = tangent;	vec3 B = cross(T, N);	TBNMatrix = gl_NormalMatrix * mat3(T, B, N);//Now TBNMAtrix is expressed in camera coordinates system	/*	A more correct (I think) but slower Alternative is:	vec3 N = normalize(gl_NormalMatrix * gl_Normal);	vec3 T = normalize(mat3(gl_ModelViewMatrix) * tangent);	vec3 B = cross(T, N);	TBNMatrix = mat3(T, B, N);	*/}

uniform sampler2D texture;uniform sampler2D normalmap;varying mat3 TBNMatrix;void main(void){	vec4 texture = texture2D(texture, vec2(gl_TexCoord[0]));	vec3 bumpnormal = 2.0*texture2D(normalmap, vec2(gl_TexCoord[0])).xyz - vec3(1.0);	bumpnormal=TBNMatrix*bumpnormal;//here bumpnormal is transformed from tangent to camera coordinates system	vec3 lightDirGlobal = vec3(1.0, 0.0, 0.0);//It's fixed with respect to the camera	float strength = max(dot(bumpnormal, lightDirGlobal), 0.0);		gl_FragColor = vec4(strength * texture.rgb, 1.0);}

EDIT: If you still want to send the light direction to the fragment shader (an use less varyings) because the TBNMatrix is orthogonal.
attribute vec3 tangent;vec3 lightDirGlobal = vec3(1.0, 0.0, 0.0);//Fixed w.r.t the cameravarying vec3 lightDirTangential;void main(void){	gl_Position = ftransform();	gl_TexCoord[0] = gl_MultiTexCoord0;	vec3 N = gl_Normal;	vec3 T = tangent;	vec3 B = cross(T, N);	TBNMatrix = gl_NormalMatrix * mat3(T, B, N);//Now TBNMAtrix is expressed in camera coordinates system	/*	A more correct (I think) but slower Alternative is:	vec3 N = normalize(gl_NormalMatrix * gl_Normal);	vec3 T = normalize(mat3(gl_ModelViewMatrix) * tangent);	vec3 B = cross(T, N);	TBNMatrix = mat3(T, B, N);	*/	lightDirTangential = lightDirGlobal * TBNMatrix;//equivalent to: transpose(TBNMatrix) * lightDirGlobal;}

uniform sampler2D texture;uniform sampler2D normalmap;varying vec3 lightDirTangential;void main(void){	vec4 texture = texture2D(texture, vec2(gl_TexCoord[0]));	vec3 bumpnormal = 2.0*texture2D(normalmap, vec2(gl_TexCoord[0])).xyz-vec3(1.0);	float strength = max(dot(bumpnormal, lightDirTangential), 0.0);		gl_FragColor = vec4(strength * texture.rgb, 1.0);}

[Edited by - knighty on April 18, 2009 10:10:50 AM]

Share on other sites
Wow, thanks a lot, knighty!
I tried your last version (since i think it will be the fastest) and it works! And more important, thanks to your explanations, i think i understand it :)

Share on other sites
Oh, one more question:
Quote:
 1) because you transformed the vertex into the camera's coordinates system, you have to do the same with the TBN matrix.

I didn't transform the vertex or the tangent into the camera's coordinate system consciously. In my application, vertices and tangents are calculated in the "world" coordinate system, so where are they transformed? Is this done automatically by OpenGL?

Share on other sites
Yes, you are right. It's faster. Finally, it's just a modification of your initial code: transformation of normal and tangent vectors to camera space, reversing the multiplication of lightDirGlobal by TBNMatrix and normalizing the normal map's texel. It's all about doing things in the right coordinates system [smile].

Share on other sites
Quote:
Original post by gagabla
Oh, one more question:
Quote:
 1) because you transformed the vertex into the camera's coordinates system, you have to do the same with the TBN matrix.

I didn't transform the vertex or the tangent into the camera's coordinate system consciously. In my application, vertices and tangents are calculated in the "world" coordinate system, so where are they transformed? Is this done automatically by OpenGL?

gl_Position = ftransform();
is "equivalent" to:
gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;

Edit:{
When you do:
vec3 N = gl_NormalMatrix * gl_Normal;
You are asking OpenGL to transform the normal from the Model coordinates system to camera coordinates system. gl_NormalMatrix is the inverse transpose of the "vectorial" part of the gl_ModelViewMatrix.
}

In OpenGL there is no distinction between model,world and view space. They are packed together. It's up to the programmer to decide where the camera transformation stops and when the model transformation begins. Confusing! But with a little effort (and many exercices[wink]) everey thing will be clear.

It would take too long to explain here. It is explained better than I could ever do in the red book especially the 3rd chapter.

[Edited by - knighty on April 18, 2009 11:02:23 AM]

Share on other sites
But gl_Vertex and gl_Normal are always in the global coordinate system, aren't they?

I will have a look at the red book, now i am too confused to discuss about it :D

EDIT:
Looking at the following two lines of the vertex shader:
TBNMatrix = gl_NormalMatrix * mat3(T, B, N);//Now TBNMAtrix is expressed in camera coordinates systemlightDirTangential = lightDirGlobal * TBNMatrix;//equivalent to: transpose(TBNMatrix) * lightDirGlobal;

What transformation does TBNMatrix (as is) stand for?
From camera space, to tangent space?
Then, what does transpose(TBNMatrix) * XXX do?
From Tangent space to camera space?

Share on other sites
Quote:
 Original post by gagablaBut gl_Vertex and gl_Normal are always in the global coordinate system, aren't they?I will have a look at the red book, now i am too confused to discuss about it :D

LOL!(Edit: about you being confused... Don't worry! coordinates system and transformations are perhaps the most confusing stuff in comuter graphics but then you will see that it's not that hard!)

[Edited by - knighty on April 18, 2009 12:38:20 PM]

Share on other sites
Quote:
 Original post by gagablaBut gl_Vertex and gl_Normal are always in the global coordinate system, aren't they?I will have a look at the red book, now i am too confused to discuss about it :DEDIT:Looking at the following two lines of the vertex shader:*** Source Snippet Removed ***What transformation does TBNMatrix (as is) stand for?From camera space, to tangent space?Then, what does transpose(TBNMatrix) * XXX do? From Tangent space to camera space?

What did we do!
1) we had the normal and a tangent vector in the model coordinates system (C.S.).
2) we calculated the binormal in the model C.S. also.
3) we constructed the TBN Matrix out of tangent, binormal and normal vectors.
4) we transformed the TBN Matrix to the camera C.S. Now the 3 columns of TBNMatrix are actually the tagent, binormal and normal vector expressed in the camera C.S.
Now remember that TBNMatrix is orthogonal, that is if you multiply it by its transpose you will get the identity! so we don't have to do weired math to get it's inverse: it's simply it's transpose!
4) we transformed the light direction from the camera C.S. to tangent space. How? We did:
lightDirTangential = lightDirGlobal * TBNMatrix;
That is equivalent to:
lightDirTangential = transpose(TBNMatrix) lightDirGlobal;
That is equivalent to:
lightDirTangential = Inverse(TBNMatrix) lightDirGlobal;//Inverse doesn't exist in GLSL v1.2 and I don't know if it does in more recent versions.

In GLSL, doing:
vector * matrix
is equivalent to:
transpose(matrix)*vector
But the former is faster!

Share on other sites
Quote:
 Original post by knighty4) we transformed the light direction from the camera C.S. to tangent space...

Ok, so the light, i wanted to come globaly from the right now comes from the camera? But in Lumina it looks as if it realy came from right, is this simply because the camera in Lumina looks along the global X-Axis?

In my application, the shaders sadly don't simply work as they are now. Somehow the light (as said, should be global, my "sun") depends on my camera orientation. But for *some* triangles in *some* camera positions, one could believe to see something like bumpmapping :D

Thanks for your help and explanations so far!!!

Share on other sites
Quote:
Original post by gagabla
Quote:
 Original post by knighty4) we transformed the light direction from the camera C.S. to tangent space...

Ok, so the light, i wanted to come globaly from the right now comes from the camera? But in Lumina it looks as if it realy came from right, is this simply because the camera in Lumina looks along the global X-Axis?

It comes from the right because it's direction is (1.0,0.0,0.0) with respect to the camera. Every thing is all right.
In camera coordinates system, Z axis goes from the eye (ouch!) then behind. X axis goes to the right and Y is up.
Quote:
 In my application, the shaders sadly don't simply work as they are now. Somehow the light (as said, should be global, my "sun") depends on my camera orientation. But for *some* triangles in *some* camera positions, one could believe to see something like bumpmapping :DThanks for your help and explanations so far!!!

I'm not sur I understand. Do you want the light to be fixed w.r.t. the model? in this case you will have to do:
TBNMatrix = mat3(T, B, N);// no need to multiply by gl_NormalMatrix

But then you will say again: "the bumpmapping does not update"! This is natural
because you are doing diffuse lighting. If you add specular hilights things would become more interresting.

Here is an very good tutorial about normal mapping.

Share on other sites
The light direction should be fixed in the world system. In my application this means, that bumpmapping will not change when moving around (and thats ok). But i want the bumpmapping to update, when my object is moving in the world.
This is not the same as simple diffuse lighting (as i understand it) because my normal map is influencing the lightning additionaly (resulting in a higher "fake" level of details).
In my Lumina example, the object is a rotating torus, so i thought, that bumpmapping should update, event for a "global" light from right.

I tried your solution without the gl_NormalMatrix transformation of the TBN-Matrix. In Lumina, i get again the rotating torus with the initial bumpmapping (perhaps Lumina rotates somehow the world and not the torus? This would explain this result at least). In my application nothing comparable to bumpmappping is visible, its more like an overlay of both textures.

Share on other sites
I see!
specifications:
1) the light direction is defined in "world" space.
2) the model may be moved, rotated...etc in the world space.
3) the camera may also be moved and rotated in the world space.
In your application you generaly do something like this:
- transform the view-->let's call the resulting matrix "world to view matrix"
- transform the model-->the resulting matrix is now the modelviewmatrix
- draw the model

In your vertex shader you need to transform the light's direction into tangent space. You don't transform TBNMatrix so it stays in the model space. You have to transform the light's direction into the model space first. To do that you need the "world to model matrix" that is the inverse of the "model to world matrix".[grin]

How to retrive that evil matrix?
In your application (again) you will have to save the "model to world matrix" and then invert it. So you will do:
- Transform the view
- Save the resulting matrix with glGetFloatv(GL_MODELVIEW_MATRIX,world2view)
- glPushMatrix()
- Transform the model
- Save the resulting matrix with glGetFloatv(GL_MODELVIEW_MATRIX,model2world);//You dont need the translation and projective part of that matrix so it's better to extract the 3x3 upper left part of it.
- invert model2world-->world2model (if you did just translation and/or rotation the inverse will be equal to the transpose)
- glPopMatrix()
- glMultMatrix(model2world)
- draw the model

Then, in the vertex shader yo do:
uniform mat3 world2model;attribute vec3 tangent;vec3 lightDirGlobal = vec3(1.0, 0.0, 0.0);//It would be better as an uniformvarying vec3 lightDirTangential;void main(void){	gl_Position = ftransform();	gl_TexCoord[0] = gl_MultiTexCoord0;	vec3 N = gl_Normal;	vec3 T = tangent;	vec3 B = cross(T, N);	TBNMatrix = mat3(T, B, N);	vec3 lightDirInModelSpace = world2model * lightDirGlobal;	lightDirTangential = lightDirInModelSpace * TBNMatrix;}

Ouf!

[Edited by - knighty on April 19, 2009 7:51:52 AM]

Share on other sites
You are the man, knighty!
For the first time, i have seen some real bumpmapping in my application. Up to now only on the enviroment (which is not moving, but can be transformed by rockets), but i will now try to apply it to the character. But this will take some time, since i have to adjust a lot, to calculate the tangents for the characters triangles (up to now, its a framework class, that generates the character). I will post screenshots and code soon!

Thanks for your dazzling help, knighty.

Share on other sites
Quote:
 Original post by gagablaYou are the man, knighty!For the first time, i have seen some real bumpmapping in my application. Up to now only on the enviroment (which is not moving, but can be transformed by rockets), but i will now try to apply it to the character. But this will take some time, since i have to adjust a lot, to calculate the tangents for the characters triangles (up to now, its a framework class, that generates the character). I will post screenshots and code soon!Thanks for your dazzling help, knighty.

Thak you too! I'm looking forward for your screenshots.

Share on other sites
Phew, getting over my own stupidity it works now :)
I uploaded a screenshot, its the last image on the Assets page.
Ignore the ugly character in the front, i hate modelling.
The dint in the terrain has been generated at runtime, and the bumpmapping updated correctly!

Next step are explosions and a postprocessing glow filter. See you soon :D

Share on other sites
Good work, and nice picture there!