Tangent Space Mayhem

424

October 13, 2005 02:12 AM

In that case, it looks like your source model is the problem. Are you definitely sure that your source model is okay? If you use 3dsMax make sure that you don't have different smoothing groups applied to your mesh (ie. select all your faces using an edit-mesh modifier and apply a single smoothing group to your faces). If you're using Maya, make sure you soften the edges completely. Not sure how this is handled in other packages like Blender or whatever you're using...

Quote:(normals are indexed in the face corners and not the same as positions).

This is most likely your problem. Your source model probably has 2 neighbouring triangles (ie. triangles that share an edge or vertex, but the normal is different for each triangle at the same vertex position). You need to apply corrections to your source model so that you have only 1 normal per vertex (across all faces that share the vertex). Normal mapping doesn't work nicely when you use multiple smoothing groups.

Good luck.

do unto others... and then run like hell.

RPTD

359

Author

October 13, 2005 03:30 AM

nope, nope... that is not the problem

i use blender3d and an export script i wrote myself. edge creasing is used for normal generation which blows smoothing groups out of the water (sorry, it's just that this is the not-countable-many-time i run across people with problems with SGs, hence i chose a better solution).

and i said it many times before: the normals ARE ok. i checked them visually. there exists no normal discrepancy. and rendering with normals only yields good results (just no normal-mapping).

it is definitly not a normal problem, it is a tangent problem, and i can be sure it is not a calculation error as checking with the graphical debug i see clearly that normals and tangents are all pointing where they should.

EDIT: edges are what the name says, edges. split normals are only generated for faces having a hard edge and there stepping has to be visible as this is a hard edge, not a soft one where all normals are smoothed across.

Life's like a Hydra... cut off one problem just to have two more popping out.
Leader and Coder: Project Epsylon | Drag[en]gine Game Engine

FReY

424

October 13, 2005 10:22 AM

Well, the phenomenon you have on your screenshot is edge-creasing. The only way this would happen is if you have a different TBN per vertex (even if it is a wrong TBN, it will be the same and yield the same result with all faces that are using it) which means you probably have split normals/tangents at those points on the model. There is NO other way that this phenomenon can occur.

Please, post another screenshot with your normals used in conventional lighting (ie. no vertex shader, no normal mapping, no TBN, just straight fixed-function lighting.)

Quote:
i use blender3d and an export script i wrote myself. edge creasing is used for normal generation which blows smoothing groups out of the water

So *ARE* you using this 'edge creasing' on your model? If you are, it will yield the same results as smoothing groups (it's just another method of splitting normals on your faces)

I suggest you run a function like this on your models to make sure (if a face shares a vert with another face it makes sure that the normal is the same)

bool DoesModelHaveHardEdges(Model *mode){  for (i = 0; i < model->numfaces; i++)  {    for (j = i+1; j < model->numfaces; j++)    {       for (k = 0; k < 3; k++)       {         for (l = 0; l < 3; l++)         {            if (model->face.vertindex[k] == model->face[j].vertindex[l])           {             if (model->face.normalindex[k] != model->face[j].normalindex[l])             {               return true;             }           }         }      }    }  }  return false;}

Quote:
EDIT: my major problem here is also understanding the matrix. in this tutorial (http://www.blacksmith-studios.dk/projects/downloads/tangent_matrix_derivation.php) they use the 'inverse TBN', which i think i have here too, but they have 'negated' values in the matrix. reproducing those negated values though it looks even worse. i'm all out of ideas on this one, really.

Just to clarify something. The inverse of the TBN will be the transpose of the TBN if the TBN is orthonormal. This simply means the TBN rows and columns are swapped. There is no dire need to invert the matrix by using it's determinant, so there is no need to negate any values when simply a transpose will do.

do unto others... and then run like hell.

RPTD

359

Author

October 13, 2005 12:23 PM

Quote:Original post by FReY
Well, the phenomenon you have on your screenshot is edge-creasing. The only way this would happen is if you have a different TBN per vertex (even if it is a wrong TBN, it will be the same and yield the same result with all faces that are using it) which means you probably have split normals/tangents at those points on the model.

i can assure you this is not the case as i visually checked all normals. at the neck and the shoulder all normals are shared (hence faces sharing the same vertex have also the same normal)

Quote:There is NO other way that this phenomenon can occur.

i still think it is not the normal playing dull there, maybe the TBN or the shader but the normals are ok.

Quote:Please, post another screenshot with your normals used in conventional lighting (ie. no vertex shader, no normal mapping, no TBN, just straight fixed-function lighting.)

this i can not do as my graphic-module is hard-wired to shaders (i have no TLN module yet). but i messed with the shader to not transform the normal and not apply the normal map, hence lighting like always just without TBN and normal-maps.

NOTE: i disabled shadow-volumes in this one to see lighting better.

Quote:So *ARE* you using this 'edge creasing' on your model? If you are, it will yield the same results as smoothing groups (it's just another method of splitting normals on your faces)

kind of. i use it to make an edge which is hard (ergo an edge of a box for example) to be lit 'hard' with separate normals and all other edges have smoothed normals. it is much easier to work with edge creasing than smoothing groups, especially since blender has separate edge selection and handling mode.

Quote:I suggest you run a function like this on your models to make sure (if a face shares a vert with another face it makes sure that the normal is the same)

*** Source Snippet Removed ***

no-go. like i mentioned above there are edges which have to be hard to be lit properly. those are located at places with sharp angles like the mouth or horns and claws. the parts having problems with lighting all have no hard edges and hence all the normals there are shared. that code will hence bounce out on every model i have.

Quote:Just to clarify something. The inverse of the TBN will be the transpose of the TBN if the TBN is orthonormal. This simply means the TBN rows and columns are swapped. There is no dire need to invert the matrix by using it's determinant, so there is no need to negate any values when simply a transpose will do.

sounds logical to me. still there has to be something wrong with the TBN or the fragment program. if you pay attention to the hind feet in both screenshots you will notice that in the normal only both hind feet are totally unlit, because they are outside the light range. in the TBN version though they receive utterly muich light which can not be correct. as in both cases i use the very same VBOs (hence the same normals and tangents) i assume that those are not incorrect as otherwise in both screens it should look wrong.

EDIT: i'm not sure about it but maybe the lighting with normals only is not fully correct too. if i am not mistaken then the edges of the faces have correct lighting but the pixels inside the triangle not. this would mean the interpolated normal (lightPos - inPos) is off somehow. as far as my math goes using a shorter normal inside the triangle should result in less light intensity. maybe it is effectivly not the TBN but the fragment shader having an error somewhere (if i'm correct).

still doesn't give me a hint why with TBN it is totally off.

EDIT: EDIT: made a mistake in the quickly changed shader for showing normal lighting only, i forgot the normalizing. updated the second screenshot with the corrected one. here you see that normals do work well. there are some spots i am not sure what's up there but in general it is looking ok.

EDIT: EDIT: EDIT: i played around with the scripts as i have a suspect. i set attenuation fix to 0.5 instead of calculating it per-vertex. in this setup the model is properly lit like in the normal-only screenshot, just with a normal-map applied like it should. this result means that the fragement program is working well but that the normal spit out by the TBN is utterly wrong. as i have only ommited the TBN compared to the normal-only version of my script it is clear that the TBN calculation has to be buggy and misshaping my light normals. now what can i do against that?

[Edited by - RPTD on October 13, 2005 1:23:35 PM]

Life's like a Hydra... cut off one problem just to have two more popping out.
Leader and Coder: Project Epsylon | Drag[en]gine Game Engine

FReY

424

October 14, 2005 02:42 AM

When you posted the second screenshot, did you use the exact normal from the TBN for lighting?

Other than that, I'm kinda out of ideas, except for

1.) Just out of curiosity. Use your vertex normal and a constant vector (eg. [0,1,0]) to calculate your bitangent with a cross-product and then re-calculate your tangent from there with another cross product. Sure you will get problems where the normal and constant vector are almost parallel, but see whether a TBN generated like this (just for testing) will eliminate your artifacts in the areas you mentioned. If it doesn't fix the artefacts then you know that the TBN is not your problem, if it does fix the problem you know the TBN is your problem.

2.) You've checked that you're not overwriting any array bounds anywhere or using 32 bit indices in a 16 bit array. (Silly, I know, but it can happen)

3.) Have you tried producing the entire TBN on the CPU (just so you can debugger-verify that these things are being generated properly)

do unto others... and then run like hell.

RPTD

359

Author

October 14, 2005 08:36 AM

Quote:Original post by FReY
When you posted the second screenshot, did you use the exact normal from the TBN for lighting?

instead of sending the normal through the TBN and into the fragment program i directly 'MOV'ed it into the fragment program, hence bypassing the TBN.

Quote:Other than that, I'm kinda out of ideas, except for

1.) Just out of curiosity. Use your vertex normal and a constant vector (eg. [0,1,0]) to calculate your bitangent with a cross-product and then re-calculate your tangent from there with another cross product. Sure you will get problems where the normal and constant vector are almost parallel, but see whether a TBN generated like this (just for testing) will eliminate your artifacts in the areas you mentioned. If it doesn't fix the artefacts then you know that the TBN is not your problem, if it does fix the problem you know the TBN is your problem.

this does not solve anything. i get a similar output like above with normal-lighting only. i am still sure the TBN is the problem, below i give my thoughts about what i think is the problem.

Quote:2.) You've checked that you're not overwriting any array bounds anywhere or using 32 bit indices in a 16 bit array. (Silly, I know, but it can happen)

i don't think so as normal-only works. and in this one i use length calculation which would be horribly off if a normal is off.

Quote:3.) Have you tried producing the entire TBN on the CPU (just so you can debugger-verify that these things are being generated properly)

nope but i can write a small test-app in which i calculate the TBN myself checking the output because...

... my theoris.

after fiddling around yesterday evening with my shader i think i found out what the problem is, but i am not sure. if i fixed the attenuation factor to 0.5 instead of calculating it in the fragment shader i received the same as with normal-only above. this tells me that the attenuation is wrong in the TBN version. attenuation depends though only on the length of the vector from light source to my render point, not the direction of the normal. hence if the attenuation is off, the length of the normal is off. as the only transformation that took place has been the TBN it has to be the problem. hence i have now the following situation:

before TBN: |normal| = x
after TBN: |normal| != x

this is very bad. the length of the normal has to be different which results in wrong attenuation. maybe even the normal is a bit wrong oriented. now here again the shader code responsible for this:

Quote:# compute the 3x3 transform from tangent space to object space
## matObjTan1 = normal % tangent;
XPD matObjTan1.xyz, inNormal, inTangent;
## matObjTan2 = normal; (alias)
## matObjTan0 = matObjTan1 % normal;
XPD matObjTan0.xyz, matObjTan1, inNormal;

if i compare this with the matrix used on the tutorial page i notice one difference: the determinant is not used! i know because the normal is normalized anyways for lighting this is of no importance, but if i need the length of the normal this will totally screw it.

now the first idea would be to fully construct the matrix, which though requires a lot more shader opcodes. this can not be the solution. but storing the length before the TBN and feed this to the fragment shader will not work neither as then the length is interpolated, which is no more the real normal length inside the face.

any ideas how i can solve this problem without constructing a full TBN matrix GPU side or CPU side (with transfering it afterwards to the GPU)?

Life's like a Hydra... cut off one problem just to have two more popping out.
Leader and Coder: Project Epsylon | Drag[en]gine Game Engine

RPTD

359

Author

October 16, 2005 08:29 AM

small addition. smoothing out the tangents solved the problem with the lighting on the hind-feet. still leaves a problem at places where faces of different orientation meet, like the inside of the leg meeting the belly. at this point i receive zero-tangents during the interpolation step, which is annoying. need to figure out a way to deal with that.

still there is the TBN error though. the vector from inPos to lightPos is still longer after sending it through TBN than before resulting in the other problems i had. still not found out why the TBN stretches my vectors thogh <.=.<

Life's like a Hydra... cut off one problem just to have two more popping out.
Leader and Coder: Project Epsylon | Drag[en]gine Game Engine

FReY

424

October 17, 2005 03:09 AM

Quote:
any ideas how i can solve this problem without constructing a full TBN matrix GPU side or CPU side (with transfering it afterwards to the GPU)?

Most people use a normalization cube map to sort this out. Treat your light-direction vector as a 3d texcoord and lookup it's length in a cubemap that effectively normalizes the direction vector. This is done in the pixel shader so that it normalizes the interpolated light direction vector before dotting it with normal obtained from your normal-map.

So to summarize you need to do something like this:

CPU:
* Upload object space light direction to vertex constant memory
* DrawMesh (upload vertices, texcoords, normals, tangents)

VertexShader:
* Calc Bitangent, hence TBN
* Convert object-space light direction to tangent space (using TBN)
* outputting tangent space light direction to an output texcoord
* calculate attenuation based on object space light direction.

PixelShader:
* Load interpolated tangent space light direction
* Normalize it (with RSQRT/Normalization Cube Map)
* Dot with normal obtained from normal map
* etc...

Quote:
small addition. smoothing out the tangents solved the problem with the lighting on the hind-feet.

What do you mean by 'smoothing' out the tangents? How did you do this?

Quote:
still leaves a problem at places where faces of different orientation meet, like the inside of the leg meeting the belly. at this point i receive zero-tangents during the interpolation step, which is annoying. need to figure out a way to deal with that.

Tangent interpolation step? Do you mean the per-pixel tangent interpolated across the triangle? This is not the typical way of doing bumpmapping. The normal way is mentioned above... (unless I've misunderstood something here)

do unto others... and then run like hell.

RPTD

359

Author

October 17, 2005 07:47 AM

Quote:Original post by FReYSo to summarize you need to do something like this:

CPU:
* Upload object space light direction to vertex constant memory
* DrawMesh (upload vertices, texcoords, normals, tangents)

VertexShader:
* Calc Bitangent, hence TBN
* Convert object-space light direction to tangent space (using TBN)
* outputting tangent space light direction to an output texcoord
* calculate attenuation based on object space light direction.

PixelShader:
* Load interpolated tangent space light direction
* Normalize it (with RSQRT/Normalization Cube Map)
* Dot with normal obtained from normal map
* etc...

i have all that and that is not the problem. lighting without TBN works. the problem is that the normal coming out of the TBN has to be of different length thant before entering the TBN matrix and hence attenuation calculation is horribly off. i need to fix that problem.

Quote:What do you mean by 'smoothing' out the tangents? How did you do this?

the same as with normal-smoothing. summing up tangents of neighbor faces (if not part of a hard-edge) and averaging it.

Quote:Tangent interpolation step? Do you mean the per-pixel tangent interpolated across the triangle? This is not the typical way of doing bumpmapping. The normal way is mentioned above... (unless I've misunderstood something here)

what i mean is the folliwing. imagine the belly has due to the texture its tangent running towards the tail tip. the neighbor faces of the leg have due to the texture its tangent running the opposite way. now in the vertex shader at two different vertices it happens that the two tangents in those points face opposite directions. now in stupid cases it happens that the interpolated light vector becomes 0 which results in a 'hot-spot' on the model. happens wherever faces of different tangent orientation meet (of extremly different orientation). laying out the textures in a better way should solve this but i do not want to force my modellers to know about tangent-orientation problems during skinning.

BTW: interpolating the attenuation gives me horrible results as currently there are rather large tris in the map (no tesselate algorithme yet). i do attentuation in the pixel shader which works if the normal is not of wrong length (which the TBN does for some reason)

Life's like a Hydra... cut off one problem just to have two more popping out.
Leader and Coder: Project Epsylon | Drag[en]gine Game Engine

RPTD

359

Author

October 20, 2005 09:14 AM

i really need somebody who can help me with that TBN matrix. i tried now doing debug calculations in my code instead of the shader to get readable output. i don't know why but the TBN is just totally useless. i can plug in any TBN i found so far on the net and it just does not work.

my test setup is creating the TBN, using the normal as the test vector, which should then become (0,0,1) as this is the definition of the TBN.

but i get anything, just not what i should:

Quote:[NORMAL] len1(1) len2(1) dot1(1) dot2(0.999997)
[NORMAL] len1(1) len2(1) dot1(1) dot2(0.405146)
[NORMAL] len1(1) len2(0.974384) dot1(1) dot2(0.374265)
[NORMAL] len1(1) len2(0.974384) dot1(1) dot2(0.374265)
[NORMAL] len1(1) len2(0.974384) dot1(1) dot2(0.374265)
[NORMAL] len1(1) len2(1) dot1(1) dot2(-0.996501)
[NORMAL] len1(1) len2(0.974384) dot1(1) dot2(0.374265)
[NORMAL] len1(1) len2(0.974384) dot1(1) dot2(0.374265)
[NORMAL] len1(1) len2(0.974384) dot1(1) dot2(0.374265)
[NORMAL] len1(1) len2(1) dot1(1) dot2(-0.996501)

len1 is the length of the normal i throw in, len2 the length of the normal coming out of the TBN. dot1 is normal dot normal hence always 1 and dot2 is transformed-normal * (0,0,1) which should be 1.

for testing i tried this:

Quote: normal = pComp->GetNormal( face->normal1 );
tangent = pComp->GetTangent( face->tangent1 );
decVector bt = normal % tangent;
decVector t = bt % normal;
decMatrix m;
m.a11 = t.x; m.a12 = -bt.x; m.a13 = normal.x; m.a14 = 0;
m.a21 = -t.y; m.a22 = bt.y; m.a23 = -normal.y; m.a24 = 0;
m.a31 = t.z; m.a32 = -bt.z; m.a33 = normal.z; m.a34 = 0;
m.a41 = 0; m.a42 = 0; m.a43 = 0; m.a44 = 1;
m *= 1.0f / ( t.x * t.x + bt.x * bt.x + normal.x * normal.x );
decVector n1 = normal;
decVector n2 = m * n1;
float len1 = n1.Length(), len2 = n2.Length();
decVector t1 = n1, t2 = n2;
try{
t1.Normalize(); t2.Normalize();
float dot1 = t1 * normal, dot2 = t2 * decVector(0,0,1);
printf("[NORMAL] len1(%g) len2(%g) dot1(%g) dot2(%g)\n",
len1, len2, dot1, dot2);
}catch(duException){ }

i am totally out of ideas on this one and it drives me crazy as without proper lighting it looks shit and droping normal mapping just because of this stupid TBN sux too.

[Edited by - RPTD on October 20, 2005 9:14:11 AM]

Life's like a Hydra... cut off one problem just to have two more popping out.
Leader and Coder: Project Epsylon | Drag[en]gine Game Engine

Tangent Space Mayhem

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Tangent Space Mayhem

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines