# Tangent Space Mayhem

This topic is 4469 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

reading through tons of tutorials and sample code i tried everything but that stupid Tangent Space Matrix (TBN) does simply not work and i have no clue why. the current thing i have does this: (vertex program)
Quote:
 # compute the 3x3 transform from tangent space to object space ## matObjTan1 = normal % tangent; XPD matObjTan1.xyz, vertex.normal, vertex.texcoord[1]; ## matObjTan2 = normal; (alias) ## matObjTan0 = matObjTan1 % normal; XPD matObjTan0.xyz, matObjTan1, vertex.normal;
both the incoming normal and tangent are |x|=1 . my problem is that still it seems to not work. as trial-and-error gave me no solution i need somebody to give me a hint. thus what is true? 1) my code is correct (hence the error is somewhere else) 2) my code is not correct (hence what is wrong with it?) EDIT: my major problem here is also understanding the matrix. in this tutorial (http://www.blacksmith-studios.dk/projects/downloads/tangent_matrix_derivation.php) they use the 'inverse TBN', which i think i have here too, but they have 'negated' values in the matrix. reproducing those negated values though it looks even worse. i'm all out of ideas on this one, really.

##### Share on other sites
Although I've been using GLSL rather than the ASM shaders, I was having what sounds like a similar problem a while back. You might find it helpful too look through the post I made about it:

http://www.gamedev.net/community/forums/topic.asp?topic_id=347799

(EDIT: Please excuse the broken images. My webhost apparently decided to give me the finger, and I'm not quite sure why... *grumble*)

##### Share on other sites
nice try but that is not my problem. mirrored UVs do not seem to be a problem for me as i wrote a special 'file format' to deal with this in a very easy (for both coder and artist) way.

my problem is that using the shader i get strange lighting problems.

as you can see the tangent seems to be mal-used. the normals and the lighting equations are all ok. if i render only with normals not using normal-mapping and TBN it looks ok (as ok as it can look with gourad shading only). using TBN and company things are broken.

##### Share on other sites
maybe there is problem in ur model's vertices or maybe there is problem in ur calculating the normals ...
try simple modle like sphere .
cya,
ff8

##### Share on other sites
I have to agree with ff8, but your normals look fine, just flat, try smoothing them out.

##### Share on other sites
Actually the toes look like the normals are a little screwed ;)

##### Share on other sites
Hm... First of all I want to compliment you on the image. Looks quite good, even with a few errant normals!

Second, are you orthogonalizing the tangents with the normal? I got similar artifacts when I disabled orthogonalization in my code.

//Orthoganalize the vectorscVect3 tmpTan = curVert.tangent;cVect3 tmpNorm = curVert.normal;cVect3 tmpBitan = curVert.bitangent;cVect3 newT = tmpTan -  ( tmpNorm * tmpNorm.Dot(tmpTan) );cVect3 newB = tmpBitan - ( tmpNorm * tmpNorm.Dot(tmpBitan) ) - ( newT * newT.Dot(tmpBitan) );newT.Normalize();newB.Normalize();curVert.tangent = newT;curVert.bitangent = newB;

##### Share on other sites
@all: nope, the normals are ok. turning on normal/tangent display shows both normals and tangents are perfect. furthermore i said that with normals only things look like they should (with normals only). hence i am 100% sure the normals are not the problem

@Toji: thx for the flowers... but there is more wrong with the graphics than just the tangent-problem (namely, shadow-volumes still create lot of troubles).

what i do is the following.

for my model i calculate on each frame (i don't know if 'skinning' normals/tangents is that much of a good idea speed wise) a face tangent and normals (normals are indexed in the face corners and not the same as positions). those normals and tangents are then normalized. i checked them beeing ok with graphical output not shown on the image and they are perfect.

the face tangent is calculated using (i think the same forumale you use) using a formula from this page with a slight alteration (which deals with the 'mirroring' problem).

now for the corners this yields a normal/tangent pair which is not necessary orthogonal, hence i orthogonalize them in my vertex-shader (to not waste CPU time on this one).

in general the algorithme seems to work as the terrain patches (the wall and ground) do look ok and have the normal-maps working. for those patches i use the same approach for calculating the tangents as for my objects.

now what i don't get is though why the output looks so crappy. if you look closely you see that the foot is outside the light range (ground is only ambient light), but still receives nearly full light strength at some places. so far i have seen this phenomena only if the TBN matrix is not normalized, but this is what i don't get. my input normal and tanget are both normalized and crossing them should yield again vectors of unit length. i also normalized the matrix explecitly afterwards and still have the same effect.

my vertex-program does this (beware, comments are old and off at some places).
!!ARBvp1.0## OpenGL Graphic Module Vertex Program## Copyright (C) 2004, Plüss Roland ( roland@rptd.dnsalias.net )# # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either # version 2 of the License, or (at your option) any later # version.## This program is distributed in the hope that it will be useful,# but WITHOUT ANY WARRANTY; without even the implied warranty of# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the# GNU General Public License for more details.# # You should have received a copy of the GNU General Public License# along with this program; if not, write to the Free Software# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.## input data##############ATTRIB inPos = vertex.position;					# position object spaceATTRIB inColor = vertex.color.primary;			# white (+ alpha)ATTRIB inTexPos = vertex.texcoord[0];			# texture coordinatesATTRIB inNormal = vertex.normal;				# face normal object spaceATTRIB inTangent = vertex.texcoord[1];			# face tangent in object space# output data###############OUTPUT outHPos = result.position;				# homogenous positionOUTPUT outColor = result.color.front.primary;	# color * (ambient + emissive)OUTPUT outTexPos = result.texcoord[0];			# texture coordinatesOUTPUT outLightDir = result.texcoord[1];		# light-pos tangent spaceOUTPUT outHalfDir = result.texcoord[2];			# half(lightdir+eye-pos) tangent spaceOUTPUT outAtten = result.texcoord[3];			# attenuation# constants############## eyePARAM pEyePos = program.local[0];	# eye position in object space# lightPARAM pLightPos = program.local[1];	# object spacePARAM pLightDir = program.local[2];	# object spacePARAM pLightColor = program.local[3];PARAM pLightOptions = program.local[4];	# type, power, cutoff, spotexp# opengl states#################PARAM pMatTex1[4] = { state.matrix.texture[0] };PARAM pMatMVP[4] = { state.matrix.mvp };# variables#############TEMP matObjTan0, matObjTan1, lightDir, t1, t2;ATTRIB matObjTan2 = vertex.normal;# calculate texture coordinates## outTexPos = oglMatTex1 * inTexPosDPH outTexPos.x, inTexPos, pMatTex1[0];DPH outTexPos.y, inTexPos, pMatTex1[1];# color is in_color## outColor = inColor;MOV outColor, inColor;# compute the 3x3 transform from tangent space to object space## matObjTan1 = normal % tangent;XPD matObjTan1.xyz, vertex.normal, vertex.texcoord[1];## matObjTan2 = normal; (alias)## matObjTan0 = matObjTan1 % normal;XPD matObjTan0.xyz, matObjTan1, vertex.normal;# calculate light vector## outLightDir.xyz = matObjTan * (lightPos - inPos).xyz;SUB t1, pLightPos, inPos;DP3 lightDir.x, matObjTan0, t1;DP3 lightDir.y, matObjTan1, t1;DP3 lightDir.z, matObjTan2, t1;MOV outLightDir, lightDir;DP3 lightDir.w, lightDir, lightDir;MOV outAtten.x, lightDir.w;RSQ lightDir.w, lightDir.w;MUL lightDir.xyz, lightDir, lightDir.w;# half light-view vector## outHalfDir.xyz = ( outLightDir.xyz + (matObjTan * (eyePos.xyz-inPos.xyz) ) * 0.5;SUB t1, pEyePos, inPos;DP3 t2.x, matObjTan0, t1;DP3 t2.y, matObjTan1, t1;DP3 t2.z, matObjTan2, t1;DP3 t2.w, t2, t2;RSQ t2.w, t2.w;MAD t1.xyz, t2, t2.w, lightDir;MUL outHalfDir.xyz, t1, 0.5;# attenuation## outAtten.x = outLightDir.xyz * outLightDir.xyz;# done above# transform the vertex with the matrix## outHPos = oglMatProj * (oglMatMdl * inPos);DPH outHPos.x, inPos, pMatMVP[0];DPH outHPos.y, inPos, pMatMVP[1];DPH outHPos.z, inPos, pMatMVP[2];DPH outHPos.w, inPos, pMatMVP[3];# end of programEND

##### Share on other sites
Your tangents must be well conditioned across the triangle surface. This means that across any edge of a triangle, your tangent should not change direction very drastically (within a couple of degrees eg. 10-30 is normally fine)

A good way to visually inspect this is to render your tangents at each vertex as lines (as you would do with normals) and inspect the areas that are giving you problems. You will probably find cases where 2 neighbouring verts have wildly varying tangents.

Hope that helps...

##### Share on other sites
i rendered with normal/tangents shown and in the the problem areas outlined the tangents are (after normalizing) less than 1° apart from each other. i don't think that's the problem.

##### Share on other sites
In that case, it looks like your source model is the problem. Are you definitely sure that your source model is okay? If you use 3dsMax make sure that you don't have different smoothing groups applied to your mesh (ie. select all your faces using an edit-mesh modifier and apply a single smoothing group to your faces). If you're using Maya, make sure you soften the edges completely. Not sure how this is handled in other packages like Blender or whatever you're using...

Quote:
 (normals are indexed in the face corners and not the same as positions).

This is most likely your problem. Your source model probably has 2 neighbouring triangles (ie. triangles that share an edge or vertex, but the normal is different for each triangle at the same vertex position). You need to apply corrections to your source model so that you have only 1 normal per vertex (across all faces that share the vertex). Normal mapping doesn't work nicely when you use multiple smoothing groups.

Good luck.

##### Share on other sites
nope, nope... that is not the problem

i use blender3d and an export script i wrote myself. edge creasing is used for normal generation which blows smoothing groups out of the water (sorry, it's just that this is the not-countable-many-time i run across people with problems with SGs, hence i chose a better solution).

and i said it many times before: the normals ARE ok. i checked them visually. there exists no normal discrepancy. and rendering with normals only yields good results (just no normal-mapping).

it is definitly not a normal problem, it is a tangent problem, and i can be sure it is not a calculation error as checking with the graphical debug i see clearly that normals and tangents are all pointing where they should.

EDIT: edges are what the name says, edges. split normals are only generated for faces having a hard edge and there stepping has to be visible as this is a hard edge, not a soft one where all normals are smoothed across.

##### Share on other sites
Well, the phenomenon you have on your screenshot is edge-creasing. The only way this would happen is if you have a different TBN per vertex (even if it is a wrong TBN, it will be the same and yield the same result with all faces that are using it) which means you probably have split normals/tangents at those points on the model. There is NO other way that this phenomenon can occur.

Please, post another screenshot with your normals used in conventional lighting (ie. no vertex shader, no normal mapping, no TBN, just straight fixed-function lighting.)

Quote:
 i use blender3d and an export script i wrote myself. edge creasing is used for normal generation which blows smoothing groups out of the water

So *ARE* you using this 'edge creasing' on your model? If you are, it will yield the same results as smoothing groups (it's just another method of splitting normals on your faces)

I suggest you run a function like this on your models to make sure (if a face shares a vert with another face it makes sure that the normal is the same)

bool DoesModelHaveHardEdges(Model *mode){  for (i = 0; i < model->numfaces; i++)  {    for (j = i+1; j < model->numfaces; j++)    {       for (k = 0; k < 3; k++)       {         for (l = 0; l < 3; l++)         {            if (model->face[i].vertindex[k] == model->face[j].vertindex[l])           {             if (model->face[i].normalindex[k] != model->face[j].normalindex[l])             {               return true;             }           }         }      }    }  }  return false;}

Quote:
 EDIT: my major problem here is also understanding the matrix. in this tutorial (http://www.blacksmith-studios.dk/projects/downloads/tangent_matrix_derivation.php) they use the 'inverse TBN', which i think i have here too, but they have 'negated' values in the matrix. reproducing those negated values though it looks even worse. i'm all out of ideas on this one, really.

Just to clarify something. The inverse of the TBN will be the transpose of the TBN if the TBN is orthonormal. This simply means the TBN rows and columns are swapped. There is no dire need to invert the matrix by using it's determinant, so there is no need to negate any values when simply a transpose will do.

##### Share on other sites
Quote:
 Original post by FReYWell, the phenomenon you have on your screenshot is edge-creasing. The only way this would happen is if you have a different TBN per vertex (even if it is a wrong TBN, it will be the same and yield the same result with all faces that are using it) which means you probably have split normals/tangents at those points on the model.

i can assure you this is not the case as i visually checked all normals. at the neck and the shoulder all normals are shared (hence faces sharing the same vertex have also the same normal)
Quote:
 There is NO other way that this phenomenon can occur.

i still think it is not the normal playing dull there, maybe the TBN or the shader but the normals are ok.

Quote:
 Please, post another screenshot with your normals used in conventional lighting (ie. no vertex shader, no normal mapping, no TBN, just straight fixed-function lighting.)

this i can not do as my graphic-module is hard-wired to shaders (i have no TLN module yet). but i messed with the shader to not transform the normal and not apply the normal map, hence lighting like always just without TBN and normal-maps.

NOTE: i disabled shadow-volumes in this one to see lighting better.

Quote:
 So *ARE* you using this 'edge creasing' on your model? If you are, it will yield the same results as smoothing groups (it's just another method of splitting normals on your faces)

kind of. i use it to make an edge which is hard (ergo an edge of a box for example) to be lit 'hard' with separate normals and all other edges have smoothed normals. it is much easier to work with edge creasing than smoothing groups, especially since blender has separate edge selection and handling mode.

Quote:
 I suggest you run a function like this on your models to make sure (if a face shares a vert with another face it makes sure that the normal is the same)*** Source Snippet Removed ***

no-go. like i mentioned above there are edges which have to be hard to be lit properly. those are located at places with sharp angles like the mouth or horns and claws. the parts having problems with lighting all have no hard edges and hence all the normals there are shared. that code will hence bounce out on every model i have.

Quote:
 Just to clarify something. The inverse of the TBN will be the transpose of the TBN if the TBN is orthonormal. This simply means the TBN rows and columns are swapped. There is no dire need to invert the matrix by using it's determinant, so there is no need to negate any values when simply a transpose will do.

sounds logical to me. still there has to be something wrong with the TBN or the fragment program. if you pay attention to the hind feet in both screenshots you will notice that in the normal only both hind feet are totally unlit, because they are outside the light range. in the TBN version though they receive utterly muich light which can not be correct. as in both cases i use the very same VBOs (hence the same normals and tangents) i assume that those are not incorrect as otherwise in both screens it should look wrong.

EDIT: i'm not sure about it but maybe the lighting with normals only is not fully correct too. if i am not mistaken then the edges of the faces have correct lighting but the pixels inside the triangle not. this would mean the interpolated normal (lightPos - inPos) is off somehow. as far as my math goes using a shorter normal inside the triangle should result in less light intensity. maybe it is effectivly not the TBN but the fragment shader having an error somewhere (if i'm correct).

still doesn't give me a hint why with TBN it is totally off.

EDIT: EDIT: made a mistake in the quickly changed shader for showing normal lighting only, i forgot the normalizing. updated the second screenshot with the corrected one. here you see that normals do work well. there are some spots i am not sure what's up there but in general it is looking ok.

EDIT: EDIT: EDIT: i played around with the scripts as i have a suspect. i set attenuation fix to 0.5 instead of calculating it per-vertex. in this setup the model is properly lit like in the normal-only screenshot, just with a normal-map applied like it should. this result means that the fragement program is working well but that the normal spit out by the TBN is utterly wrong. as i have only ommited the TBN compared to the normal-only version of my script it is clear that the TBN calculation has to be buggy and misshaping my light normals. now what can i do against that?

[Edited by - RPTD on October 13, 2005 1:23:35 PM]

##### Share on other sites
When you posted the second screenshot, did you use the exact normal from the TBN for lighting?

Other than that, I'm kinda out of ideas, except for

1.) Just out of curiosity. Use your vertex normal and a constant vector (eg. [0,1,0]) to calculate your bitangent with a cross-product and then re-calculate your tangent from there with another cross product. Sure you will get problems where the normal and constant vector are almost parallel, but see whether a TBN generated like this (just for testing) will eliminate your artifacts in the areas you mentioned. If it doesn't fix the artefacts then you know that the TBN is not your problem, if it does fix the problem you know the TBN is your problem.

2.) You've checked that you're not overwriting any array bounds anywhere or using 32 bit indices in a 16 bit array. (Silly, I know, but it can happen)

3.) Have you tried producing the entire TBN on the CPU (just so you can debugger-verify that these things are being generated properly)

##### Share on other sites
Quote:
 Original post by FReYWhen you posted the second screenshot, did you use the exact normal from the TBN for lighting?

instead of sending the normal through the TBN and into the fragment program i directly 'MOV'ed it into the fragment program, hence bypassing the TBN.

Quote:
 Other than that, I'm kinda out of ideas, except for1.) Just out of curiosity. Use your vertex normal and a constant vector (eg. [0,1,0]) to calculate your bitangent with a cross-product and then re-calculate your tangent from there with another cross product. Sure you will get problems where the normal and constant vector are almost parallel, but see whether a TBN generated like this (just for testing) will eliminate your artifacts in the areas you mentioned. If it doesn't fix the artefacts then you know that the TBN is not your problem, if it does fix the problem you know the TBN is your problem.

this does not solve anything. i get a similar output like above with normal-lighting only. i am still sure the TBN is the problem, below i give my thoughts about what i think is the problem.

Quote:
 2.) You've checked that you're not overwriting any array bounds anywhere or using 32 bit indices in a 16 bit array. (Silly, I know, but it can happen)

i don't think so as normal-only works. and in this one i use length calculation which would be horribly off if a normal is off.

Quote:
 3.) Have you tried producing the entire TBN on the CPU (just so you can debugger-verify that these things are being generated properly)

nope but i can write a small test-app in which i calculate the TBN myself checking the output because...

... my theoris.

after fiddling around yesterday evening with my shader i think i found out what the problem is, but i am not sure. if i fixed the attenuation factor to 0.5 instead of calculating it in the fragment shader i received the same as with normal-only above. this tells me that the attenuation is wrong in the TBN version. attenuation depends though only on the length of the vector from light source to my render point, not the direction of the normal. hence if the attenuation is off, the length of the normal is off. as the only transformation that took place has been the TBN it has to be the problem. hence i have now the following situation:

before TBN: |normal| = x
after TBN: |normal| != x

this is very bad. the length of the normal has to be different which results in wrong attenuation. maybe even the normal is a bit wrong oriented. now here again the shader code responsible for this:
Quote:
 # compute the 3x3 transform from tangent space to object space## matObjTan1 = normal % tangent;XPD matObjTan1.xyz, inNormal, inTangent;## matObjTan2 = normal; (alias)## matObjTan0 = matObjTan1 % normal;XPD matObjTan0.xyz, matObjTan1, inNormal;

if i compare this with the matrix used on the tutorial page i notice one difference: the determinant is not used! i know because the normal is normalized anyways for lighting this is of no importance, but if i need the length of the normal this will totally screw it.

now the first idea would be to fully construct the matrix, which though requires a lot more shader opcodes. this can not be the solution. but storing the length before the TBN and feed this to the fragment shader will not work neither as then the length is interpolated, which is no more the real normal length inside the face.

any ideas how i can solve this problem without constructing a full TBN matrix GPU side or CPU side (with transfering it afterwards to the GPU)?

##### Share on other sites
small addition. smoothing out the tangents solved the problem with the lighting on the hind-feet. still leaves a problem at places where faces of different orientation meet, like the inside of the leg meeting the belly. at this point i receive zero-tangents during the interpolation step, which is annoying. need to figure out a way to deal with that.

still there is the TBN error though. the vector from inPos to lightPos is still longer after sending it through TBN than before resulting in the other problems i had. still not found out why the TBN stretches my vectors thogh <.=.<

##### Share on other sites
Quote:
 any ideas how i can solve this problem without constructing a full TBN matrix GPU side or CPU side (with transfering it afterwards to the GPU)?

Most people use a normalization cube map to sort this out. Treat your light-direction vector as a 3d texcoord and lookup it's length in a cubemap that effectively normalizes the direction vector. This is done in the pixel shader so that it normalizes the interpolated light direction vector before dotting it with normal obtained from your normal-map.

So to summarize you need to do something like this:

CPU:
* Upload object space light direction to vertex constant memory
* DrawMesh (upload vertices, texcoords, normals, tangents)

* Calc Bitangent, hence TBN
* Convert object-space light direction to tangent space (using TBN)
* outputting tangent space light direction to an output texcoord
* calculate attenuation based on object space light direction.

* Load interpolated tangent space light direction
* Normalize it (with RSQRT/Normalization Cube Map)
* Dot with normal obtained from normal map
* etc...

Quote:
 small addition. smoothing out the tangents solved the problem with the lighting on the hind-feet.

What do you mean by 'smoothing' out the tangents? How did you do this?

Quote:
 still leaves a problem at places where faces of different orientation meet, like the inside of the leg meeting the belly. at this point i receive zero-tangents during the interpolation step, which is annoying. need to figure out a way to deal with that.

Tangent interpolation step? Do you mean the per-pixel tangent interpolated across the triangle? This is not the typical way of doing bumpmapping. The normal way is mentioned above... (unless I've misunderstood something here)

##### Share on other sites
Quote:
 Original post by FReYSo to summarize you need to do something like this:CPU: * Upload object space light direction to vertex constant memory * DrawMesh (upload vertices, texcoords, normals, tangents) VertexShader: * Calc Bitangent, hence TBN * Convert object-space light direction to tangent space (using TBN) * outputting tangent space light direction to an output texcoord * calculate attenuation based on object space light direction.PixelShader: * Load interpolated tangent space light direction * Normalize it (with RSQRT/Normalization Cube Map) * Dot with normal obtained from normal map * etc...

i have all that and that is not the problem. lighting without TBN works. the problem is that the normal coming out of the TBN has to be of different length thant before entering the TBN matrix and hence attenuation calculation is horribly off. i need to fix that problem.

Quote:
 What do you mean by 'smoothing' out the tangents? How did you do this?

the same as with normal-smoothing. summing up tangents of neighbor faces (if not part of a hard-edge) and averaging it.[/quote]

Quote:
 Tangent interpolation step? Do you mean the per-pixel tangent interpolated across the triangle? This is not the typical way of doing bumpmapping. The normal way is mentioned above... (unless I've misunderstood something here)

what i mean is the folliwing. imagine the belly has due to the texture its tangent running towards the tail tip. the neighbor faces of the leg have due to the texture its tangent running the opposite way. now in the vertex shader at two different vertices it happens that the two tangents in those points face opposite directions. now in stupid cases it happens that the interpolated light vector becomes 0 which results in a 'hot-spot' on the model. happens wherever faces of different tangent orientation meet (of extremly different orientation). laying out the textures in a better way should solve this but i do not want to force my modellers to know about tangent-orientation problems during skinning.

BTW: interpolating the attenuation gives me horrible results as currently there are rather large tris in the map (no tesselate algorithme yet). i do attentuation in the pixel shader which works if the normal is not of wrong length (which the TBN does for some reason)

##### Share on other sites
i really need somebody who can help me with that TBN matrix. i tried now doing debug calculations in my code instead of the shader to get readable output. i don't know why but the TBN is just totally useless. i can plug in any TBN i found so far on the net and it just does not work.

my test setup is creating the TBN, using the normal as the test vector, which should then become (0,0,1) as this is the definition of the TBN.

but i get anything, just not what i should:

Quote:
 [NORMAL] len1(1) len2(1) dot1(1) dot2(0.999997)[NORMAL] len1(1) len2(1) dot1(1) dot2(0.405146)[NORMAL] len1(1) len2(0.974384) dot1(1) dot2(0.374265)[NORMAL] len1(1) len2(0.974384) dot1(1) dot2(0.374265)[NORMAL] len1(1) len2(0.974384) dot1(1) dot2(0.374265)[NORMAL] len1(1) len2(1) dot1(1) dot2(-0.996501)[NORMAL] len1(1) len2(0.974384) dot1(1) dot2(0.374265)[NORMAL] len1(1) len2(0.974384) dot1(1) dot2(0.374265)[NORMAL] len1(1) len2(0.974384) dot1(1) dot2(0.374265)[NORMAL] len1(1) len2(1) dot1(1) dot2(-0.996501)

len1 is the length of the normal i throw in, len2 the length of the normal coming out of the TBN. dot1 is normal dot normal hence always 1 and dot2 is transformed-normal * (0,0,1) which should be 1.

for testing i tried this:
Quote:
 normal = pComp->GetNormal( face->normal1 ); tangent = pComp->GetTangent( face->tangent1 ); decVector bt = normal % tangent; decVector t = bt % normal; decMatrix m; m.a11 = t.x; m.a12 = -bt.x; m.a13 = normal.x; m.a14 = 0; m.a21 = -t.y; m.a22 = bt.y; m.a23 = -normal.y; m.a24 = 0; m.a31 = t.z; m.a32 = -bt.z; m.a33 = normal.z; m.a34 = 0; m.a41 = 0; m.a42 = 0; m.a43 = 0; m.a44 = 1; m *= 1.0f / ( t.x * t.x + bt.x * bt.x + normal.x * normal.x ); decVector n1 = normal; decVector n2 = m * n1; float len1 = n1.Length(), len2 = n2.Length(); decVector t1 = n1, t2 = n2; try{ t1.Normalize(); t2.Normalize(); float dot1 = t1 * normal, dot2 = t2 * decVector(0,0,1); printf("[NORMAL] len1(%g) len2(%g) dot1(%g) dot2(%g)\n", len1, len2, dot1, dot2); }catch(duException){ }

i am totally out of ideas on this one and it drives me crazy as without proper lighting it looks shit and droping normal mapping just because of this stupid TBN sux too.

[Edited by - RPTD on October 20, 2005 9:14:11 AM]

##### Share on other sites
Quote:
 Original post by FReYMost people use a normalization cube map to sort this out. Treat your light-direction vector as a 3d texcoord and lookup it's length in a cubemap that effectively normalizes the direction vector. This is done in the pixel shader so that it normalizes the interpolated light direction vector before dotting it with normal obtained from your normal-map.

I'd like to point out this is becoming less and less common, instead the normal way of doing it now is to use normal instructions as on newer cards (any recent ATI and probably NV40 and up from NV) this will be faster and doesnt use up any memory bandwidth, which is alot more presious to concerve (see GDC05 presentations linked to in the Forum FAQ, I'm pretty sure its mentioned in the GLSL one)

##### Share on other sites
Quote:
 I'd like to point out this is becoming less and less common, instead the normal way of doing it now is to use normal instructions as on newer cards (any recent ATI and probably NV40 and up from NV) this will be faster and doesnt use up any memory bandwidth

not to metnion its higher quality than textures as well

##### Share on other sites
as far as i can remeber there is no 'normalize' command in shader-assembler. at last i did not stumble across one. hence i still stick to DP3-RSQ-MUL as this works out.

##### Share on other sites
NV_fragment_program2 has the NRM instruction, and NV4x has special hardware for normalizations that runs in parallel with everything else so that normalizations are basically free. In order for the compiler to natively use NRM instead of changing it to DP3/RSQ/MUL, you need to specify half-precision (by using the NRMH instruction) and you can't write to the w component of the result. So all normalizations should look like this:
NRMH    vec2.xyz, vec1;

The compiler will also recognize DP3/RSQ/MUL sequences and change them to NRMH when possible. Be sure to declare
OPTION ARB_precision_hint_fastest;

at the top of your fragment programs, and always use write masks to limit writing only to components that you'll use later.