Skip to main content
GameDev.net gamedev.net
🔒 Locked

Mirrored UV's and Tangent Space (SOLVED)

Started by Toji Sep 26, 2005 at 1:38 PM 22 replies 30.8k views
Original Post
Toji
Toji
I've been working a lot on getting Tangent Space bump mapping working in my OpenGL renderer lately. I've got the shader for it built, got a routine to generated normals/tangents/bitangents very nicely (it's kinda like Nvidia's Mesh Mender), and I have really nice looking results if whatever you're rendering doesn't have any mirrored UV's... Unfortunately 90% of the animated models we're going to be using will be using a mirrored texture, so this is a problem. I realize fully well that the tangents and/or bitangents need to be "flipped" (inversed, in this case i think) on any triangles where the UV's are "reversed", but actually detecting that and flipping them is throwing me for a bit of a loop. First off: How do you detect if a tangent/bitangent needs to be flipped? At http://www.terathon.com/code/tangent.html they show how to calculate the "handedness" of the vector, which is simply:

tangent[a].w = (Dot(Cross(n, t), tan2[a]) < 0.0F) ? -1.0F : 1.0F;
In this case tan2 would be the bitangent, n is the normal, and t is the actual orthogonalized tangent. This makes a lot of sense (though I wonder why the don't cross the tangent and bitangent and compare that to the normal...), but doesn't seem to give me the results I would expect. In my mesh processing utility I've got it so that it generates "smoothing groups" for each vector. For the mesh normals that smoothing group is based on angle, but for the tangents and bitangents I've got it based on handedness as calculated above. This gives me two smoothing groups for each of 'em, and when processing the final, per-vertex normals I simply inverse any vectors that got put in the "-1" smoothing group. Sadly this doesn't work. For one, All of my faces seem to end up in the same group regaurdless of their orientation, but even more worrying than that is that even if I fudge the code to, say, always inverse the tangent vector, the lighting doesn't seem to change at all! This got me thinking: If the tangent is flipped, do you have the inverse the normal instead of the tangent? Everyone says that you have to flip the vectors on any mirrored UV's, but now that I think about it I don't know that I've ever heard anyone specify which vector to flip! If anyone has some insight into these questions (How do you figure out which vectors need flipping, and which ones do you flip when you do figure that out), I would be most appreciative! Thanks! [Edited by - Toji on September 30, 2005 12:47:01 AM]
// The user formerly known as Tojiro67445, formerly known as Toji [smile]
Wudan
Wudan
Hmmm mirrored UVs causing the 'feature'? That is, the 'wound' orientation on the texture is backwards from what the shader is expecting? There should be a really really fast way to determine if the texture's coords are wound backwards, IIRC OpenGl uses such a test for it's face culling. Since you're working in 2d texture space, this should be really fast, and, come to think of it, could be pre-calculated.

Lovely weather we're having :)
Toji
Toji
Ooh! Another Utah Dev-er! What part of West Valley you from? I used to live in it's neglected little brother, Magna.

I never thought about it like that, but you're right! It should be a simple matter to see if the texture is wound differently... I'll see if I can whip up something to detect that.

That still leaves me puzzled as to what exaclty you'd be flipping, though. I think I'll try the normal first, once I get the wind-direction-detection in there ^_^

Thanks! (Still open to any other suggestions...)
// The user formerly known as Tojiro67445, formerly known as Toji [smile]
Wudan
Wudan
Not having written any kind of shader or worked with bump-mapping (not to say I don't understand how it works in a fundamental way), I couldn't say how you're going to get your shader to interpret it in the desired way.

I'm from the rough part of West Valley - er, wait, that's most of it. Just down the street from the 'E' Center. Magna's pretty rough, I work in SLC though.
Toji
Toji
Quote:
Just down the street from the 'E' Center.


Ah, cool. I used to go to the theater across the street down there all the time!

Quote:
I work in SLC though.


Same, I actually live in the Avenues now. I work about a block east of the Gateway. We've probably bumped into eachother before and would never know it! Go figure. One question though: I saw on your profile your last name is Newbold. Do you just HAPPEN to have a younger brother named Marty?

In any case, on with the technical voodoo!

I messed around with my program a bit during my lunch break using a quick Doom3 importer that I whipped up just for this problem. I figured it would be best to test these things out on a format designed with normal mapping in mind. ^_^ What I found was that, interestingly enough, the normal maps looked best when I didn't do any flipping at all! I had a little "lightbulb" moment and realized that getting the correct normals has nothing to do with inverting anything, but instead is facilitated by merely splitting the mesh! The problems that occur with mirrored normal mapping are the result of forcing a single vertex to average the vectors of two opposing tangents, which can cause errors in the tangent space interpolation. If you split the mesh down the mirrored line, though, each half only interpolates with similar vectors, keeping everyone happy!

Now, I still don't have "perfect" results just yet (I think I may have mixed up my tangents and bitangents), but I'm feeling a lot closer to the final solution!

One more thing: Thank you, Wudan, for your winding suggestion. As it turns out that gave me a solution that I feel was much more "stable" than the old method that I was using. (A hacked up version of the equation in the opening post).

I'll see if I can post a screenie or two when I get results I'm satisfied with!
// The user formerly known as Tojiro67445, formerly known as Toji [smile]
SimmerD
SimmerD
In meshmender, we consider a triangle to be "flipped" in tangent space when the binormal x tangent vector is opposite to the geometric normal.

Looks like the code you posted just does a similar thing, but is comparing tangents instead of normals.

When rendering, you typically want to just use the geometric normal all the time. When smoothing, you need to detect this case if you have two geometrically neighboring triangles that have different binormals or tangents. We split these, but I suppose you could negate one of them and smooth instead.

Zemedelec
Zemedelec
Quote:
Original post by Toji


1. As Sim menioned, mirroring is detected as a sign of dot( cross( T, B ), faceNormal ).

2. Surface smoothness and tangent space smootness are different things, that must be handled differently - so N and TB should be summed and smoothed by different criteria.

3. Vertices that share mirrored and non-mirrored triangles should obviously be duplicated, BUT without losing the smoothnes of the surface!!!

So (prepare your best friend here - STL):

- compute per-face TBN and mirroring
- duplicate vertices that share differently mirrored faces
- group vertices, into set, using only their (pos,SG) - VB_n (with remapping table for original VB)
- group vertices into set, using only their (pos,SG,uv,mirror) - VB_tb (with remapping ..)
- walk the faces, and add N-s to VB_n, and TB-s to VB_ts
- orthogonalize VB_ts, by using normals in VB_n
- spread VB_tb + VB_n to your mesh. done!

P.S.: You really should want using exported normals instead of SG, so your artists can tweak them in modeling package, and if they don't want to - just get them calculated, and don't even touch SG-s.
Toji
Toji
In my case the modeler of choice for our artists is Lightwave, which apparently finds any form of normal exporting highly offensive. ^_^ I used to generate normals during the export, and even generated the tangents and bitangents during the export at one point, but it caused a few workflow issues. Namely, if the format needed to change at all or if I discovered a bug in my vector generation routines we had to re-export every single mesh again. For the artists sake, and for my own sanity while debugging, run-time normal/tangent generation is a great option. Not to mention, if I want to use a pre-existing format that doesn't contain this info (a.k.a: Doom 3), it's very easy to plug into the render pipeline.

Aside for that, though, I've still yet to figure out what exactly the problem with my tangent/bitangent routines. I had several meshes that looked "almost right", but I could still tell that something was off. It wasn't until I loaded up this head model below that I had a really good clear picture of the problem:



As you can see, the right hand side of his face looks pretty dang good. I'd go so far as to say that it's perfect. The left hand side, however, get's really messy. You can see a very clear line down the center of his face where the verticies are duplicated due to the mirrored UV's. It's worth noting that in this picture I haven't modified the "flipped" faces tangents/bitangents/normals at all. I split the verts to allow for correct interpolation, but that's it.

It's also worth noting that the vertex normals ARE properly smoothed over that center line, as you can see when we render using only the vertex normals for shading:



I'm very confused at this point. I just can't see what I'm doing wrong! The only thing that comes to mind is this: I've been reffering to algorithims (in this case the nvMeshMender source) that were designed for use in DirectX, but I'm using OpenGL. Since OpenGL interperates the "Y" texture coordinate differently than DirectX does (inversed), would that affect how you calculate the tangents?

Any insight is much appreciated, and thank you very much to everyone who has already contributed!
// The user formerly known as Tojiro67445, formerly known as Toji [smile]
Zemedelec
Zemedelec
Quote:
Original post by Toji
In my case the modeler of choice for our artists is Lightwave, which apparently finds any form of normal exporting highly offensive. ^_^ I used to generate normals during the export, and even generated the tangents and bitangents during the export at one point, but it caused a few workflow issues. Namely, if the format needed to change at all or if I discovered a bug in my vector generation routines we had to re-export every single mesh again. For the artists sake, and for my own sanity while debugging, run-time normal/tangent generation is a great option. Not to mention, if I want to use a pre-existing format that doesn't contain this info (a.k.a: Doom 3), it's very easy to plug into the render pipeline.


1. You really should have TBN precomputed, and loaded - it duplicates vertices at times! Reexporting the art shouldn't be pain, because sometimes it should be done frequently - just implement some batch export procedure, and you'll be fine and no artists will be involved - we reexport our art each build (each day), and it is very good thing.

2. Do you read my post? :)

3. It seems your TB-vectors aren't smooth at the edge - if only normal is, but TB vectors don't belong to one plane, your lighting will be incorrect.
I wrote, that you need to orthogonalize the TB vectors, by your N vector, so they become orthogonal, without changing N.
Then - how do you compute your normalmaps?
If you fail to match the TBN-basis of normalmap calculator, it can be very hard to obtain same visual quality...
rick_appleton
rick_appleton
Quote:
Original post by Toji


I'm very confused at this point. I just can't see what I'm doing wrong! The only thing that comes to mind is this: I've been reffering to algorithims (in this case the nvMeshMender source) that were designed for use in DirectX, but I'm using OpenGL. Since OpenGL interperates the "Y" texture coordinate differently than DirectX does (inversed), would that affect how you calculate the tangents?

Any insight is much appreciated, and thank you very much to everyone who has already contributed!


Disclaimer: I've never done anything like this stuff before, I only know the theory.

Take a good look at your face. If you could give us an image of only the lighting (no diffuse texture) I think you'll see what I'm seeing now. Take a close look at the nose. On the right, the light seems to be coming from the top (left?) of the image. On the left, it's coming from the bottom (maybe left, maybe right, can't really tell). This leads me to believe that one of your components is flipped. Since you mentioned the y-issue with the DirectX vs OpenGL and I'm guessing the face is uv'd right side up, I'm guessing that might indeed be your problem.
Eric Lengyel
Eric Lengyel
Hi Toji --

Which vector in the TBN basis are you flipping when the handedness is -1? It sounds like you might be flipping the tangent vector (I'm not sure), but you should flip the bitangent vector. That is, in your vertex shader, calculate B = cross(N, T.xyz) * T.w when sending in 3D normals and 4D tangents in your input arrays.

By the way, the expression Dot(Cross(n, t), b) is a scalar triple product (also the determinant of the TBN matrix). This will give you the same value for any permutation of t, b, and n that keeps them in the same order cyclicly. That is, all of the following are equivalent:

Dot(Cross(n, t), b)
Dot(Cross(b, n), t)
Dot(Cross(t, b), n)
Toji
Toji
Quote:
Original post by Zemedelec
Did you read my post? :)


Yes, I've read both of them. ^_^ Thank you fo your suggestions, but I feel like I'm already doing a mojority of what you've said (aside from pre-baking the TBN matrix, but that's a different story, and doesn't really affect the problem at hand.) Yes, I am orthogonalizing my vectors with the normal. The equation I use to orthogonalize then is pretty much a line for line duplication of nvidia's Mesh Mender method that does the same thing (with slightly different error checking that makes use of a couple of features of my vector class). I don't think that this is the problem. As for how the normal map was generated... I don't know. The model you see there is "Sarge" from Doom 3. I'm using him instead of a self generated model simply because I know that the normal map on this guy DOES work.

Quote:
Original post by rick_appleton
...This leads me to believe that one of your components is flipped.


I thought that too, but I have tried every "flipping" combination possible, and none give me correct results (most look worse. If I flip the bitangent on mirrored side it turns almost completely black.) It's interesting to note, though, that if I flip ONLY the normal on the mirrored side (which I shouldn't have to do) Then the results look as if they are correct for a light that is coming from the lower right.



Quote:
Original post by Eric Lengyel
calculate B = cross(N, T.xyz) * T.w


Though I'm not doing it in a shader, I have tried this. As I said before, it gives me nothing but a black half of a head.

I wish I had more information to give, but I'm really running out of ideas at this point. Thank you to everyone who has helped out already, and I'd love any other suggestions that anyone may have.



// The user formerly known as Tojiro67445, formerly known as Toji [smile]
skynet_
skynet_
Forget about all the flipping-stuff. Flipping is not needed at all if you generate TBN on the CPU. You only need flipping, if you generate one of the TBN vectors out of the other two in the vertex shader.

For your texturespace matrix use:
1. for N: use the averaged geometric normal
2. calculate B and T with Eric Lengyels method.
3. make B perpendicular to N
4. make T perpendicular to N
5. you don´t need (probably should not) make B and T perpendicular to each other
6. normalize T,B,N

You should average T and B of neighboured faces in step 2. Here will problems arise if your mesh has edges where faces of different texture space orientation meet. To circumvent this, you need to detect such edges and duplicate the vertices.
Toji
Toji
Well, as the result of poking around with this stupid thing I came to realize several flaws in my current rendering system. Namely: Due to the nature of the project I thought a Z-up coordinate system would be more appropriate (X to the left, Z up, Y "into" the screen), and set up my camera accordingly. It worked pretty well, but I failed to realize until my lights started moving the opposite direction of what I expected that I had inadvertently flipped my X axis due to OpenGL using a LH coordinate system >_< Because of THAT, I had also been calculating my normals inverted, and hadn't noticed because the two of them together usually gave results that looked pretty close to normal. *sigh*

Anyways, so that's all sorted out now, but I'm still having trouble with the normal maps. (It looks fairly similar to the pictures above still, just flipped to align with the new axis configuration.) After a bit more tweaking, though, I'm beginning to suspect that it may be partially rooted in my shader.

As a bit of explanation: I do some work on my program both at home and occasionally at work. My home computer has a GeForce 5500, so it can work with GLSL, but my work computer can only do fixed function. As a result, I tend to do everything so that it looks good with full shader support, but at least renders on a minimal configuration. Because of this I have several state-changes and variables in odd places to satisfy both worlds, the big key here being that I am forced to set my light after making any camera transforms due to the fact that OpenGL combines it's world and view matricies. If I set it before the camera the light wouldn't "stay put" in fixed function, but instead would follow the camera. This, of course, would screw with the light position variables in GLSL, since I was attempting to use as much of the built-in uniforms as possible.

Anyway, point being that while I had hacked out a way of making the lights stationary in both Fixed and Programmable pipes I think that the methods I used were less than "correct", and broke under odd circumstances. (Mirrored UV's being one of them.) I've since buckled down and started passing in some enviroment variables manually, including providing the shader with my own view/world matrix info that I had been tracking anyways. I haven't had a chance to test this yet, but I think it should be much improved over my old shader. I suspect I'll still have mirroring issues, but they should be a lot more workable with this setup ^_^

Here's my current (untested!) shaders. If anyone sees anything horribly wrong let me know!
//----------------// Vertex Program//----------------varying vec3 lightVec;attribute vec3 Tangent, Bitangent;uniform vec4 LightPos;uniform mat4 WorldMatrix;uniform mat4 InvWorldMatrix;   void main(){   //Pass out our transformed Position and TexCoords   gl_Position = ftransform();   gl_TexCoord[0] = gl_MultiTexCoord0;      //Calculate the Tangent-Space matrix. This may need tweaking   mat3 TBNMatrix = mat3(Bitangent, Tangent, gl_Normal);      //Get the "world position" of the vertex   vec4 vectWorld = WorldMatrix * gl_Vertex;   //Cast a ray to the light from that position   vec4 lightToPoint = LightPos - vectWorld;      //Transform the light-ray to world space   vec3 lightWorld = (InvWorldMatrix * lightToPoint).xyz;      //Put the light direction in tangent space   lightVec = TBNMatrix * lightWorld;  }//------------------// Fragment Program//------------------varying vec3 lightVec;uniform sampler2D tex, tex1;   void main(){   //Normalize the light Vector per-fragment   vec3 lightDir = normalize(lightVec);      //Unpack and normalize the per-fragment normals   vec3 cNormal = 2.0 * ( texture2D(tex1, gl_TexCoord[0].st).rgb - 0.5);   //cNormal = vec4( 0.0, 0.0, 1.0, 1.0 ); //Uncomment for per-vertex shading.   cNormal = normalize(cNormal);      //Calculate the angle to the light, clamp to 0   float angleToLight = max( dot(lightDir,cNormal), 0.0 );      //Get Diffuse texture color   vec4 cDiffuse = texture2D(tex, gl_TexCoord[0].st);   //cDiffuse = vec4( 1.0, 1.0, 1.0, 1.0 ); //Uncomment for normal shading only.      //Compute final fragment color. No ambient.    gl_FragColor = vec4(cDiffuse.rgb * angleToLight, cDiffuse.a);}


I'll post an update when I get a chance to test this stuff out. Thanks everyone for your suggestions and patience with me ^_^ It's incredible how you can program for years and still feel like a newbie at times...

[EDIT]Shader Syntatical corrections[/EDIT]

[Edited by - Toji on September 28, 2005 8:47:59 PM]
// The user formerly known as Tojiro67445, formerly known as Toji [smile]
SimmerD
SimmerD
A common debugging method with per-pixel lighting is to have a mode where you can output the normal, binormal, tangent or texcoords themselves as RGB colors. This should give you some more clues.
hplus0603
hplus0603
Quote:
I had inadvertently flipped my X axis due to OpenGL using a LH coordinate system


OpenGL uses a RH coordinate system. X is right, Y is up, Z is out (hither).

DirectX is LH; Z is in (yon).

Your images REALLY looks like the flipped tangent basis problem. Have you actually tried splitting the vertices along the flip seam, and re-generating the appropriate tangent bases? Have you tried running the mesh through MeshMender and used what that spits out, for comparision?
enum Bool { True, False, FileNotFound };
Toji
Toji
Quote:
OpenGL uses a RH coordinate system. X is right, Y is up, Z is out (hither).
DirectX is LH; Z is in (yon).


I stand corrected. Thanks ^_^

One thing I do want to reiterate, though: Yes, I AM duplicating the verticies down the mirrored seam. I'm not sure why everybody thinks I'm not, since it's something I've said that I was doing since the first post.

I guess I should take a moment to outline the process I'm going through:

-First, I pass an array of positions, UV's, and indicies into my "mesh mender".
-I do some pre-processing to generate triangle connectivity and such
-I generate a per-triangle normal and tangent/bitangent, using the methods below:
//Per Triangle Normal Generation 	cVect3 edge1 = v1.position - v0.position;	cVect3 edge2 = v2.position - v0.position;	normal = edge2.Cross( edge1 );//Per Triangle Tangents Generation        cVect3 P = v1.position - v0.position;	cVect3 Q = v2.position - v0.position;	float s1 = v1.texcoord.x - v0.texcoord.x;	float t1 = v1.texcoord.y - v0.texcoord.y;	float s2 = v2.texcoord.x - v0.texcoord.x;	float t2 = v2.texcoord.y - v0.texcoord.y;	float tmp = 0.0f;	if(fabs(s1*t2 - s2*t1) <= 0.0001f)	{		//Prevent Divide by zero funkiness		tmp = 1.0f;	}	else	{		tmp = 1.0f/(s1*t2 - s2*t1);	}		tangent.x = (t1*Q.x - t2*P.x);	tangent.y = (t1*Q.y - t2*P.y);	tangent.z = (t1*Q.z - t2*P.z);		tangent *= tmp;	bitangent.x = (s1*Q.x - s2*P.x);	bitangent.y = (s1*Q.y - s2*P.y);	bitangent.z = (s1*Q.z - s2*P.z);	bitangent *= tmp;

-The triangles are divided into smoothing groups based on a normal threshold
-Verticies that lie on a smoothing group edge are duplicated
-Per-vertex normals are calculated by summing up the normals of each triangle that shares that vertex and belongs to the same smoothing group, then normalizing
-Triangles are again divided into smoothing groups, but this time there are only two and they are determined by the handedness of the tangent and bitangent.
-Verticies are again duplicated based on smoothing group. This process is not affected by splits made in the normal smoothing process.
-Per-Vertex tangents/bitangents are calculated in the same fasion as the normals.
-The tangents are orthogonalized with the normal
-The new vertex vectors are mapped onto the original array (adding any new verts that were created) and indicies are adjusted accordingly.

Sorry if that was a little lengthy, but there simply seems to be a lot of questions about how I'm calculating this.

As for SimmerD's suggestion, I've just tried the color-by-vector approach, and it gave me some interesting results, but I want to spend some more time examining/understanding them before I post anything about it.

[EDIT]I decided to go one step futher with SimmerD's suggestion and actually drew out the vectors themselves. (I love OpenGL, this would have taken forever in DX) A few lines of debug code later and... TADA!

IMAGE LINKED FOR SIZE

Blue lines are the normals, Green are the Tangents, and Red the Bitangents. As you can see, they're all pretty consistent as they travel along the surface AND, most importantly, they're mirrored correctly! So at this point I'm absolutely certain that I've got the TBN vectors generating properly, it's just a matter of getting that to work within a shader...[/EDIT]

[Edited by - Toji on September 28, 2005 11:28:33 PM]
// The user formerly known as Tojiro67445, formerly known as Toji [smile]
Zemedelec
Zemedelec
Quote:
Original post by Toji
...
[/EDIT]


TBN looks very fine.
Just a suggestion (seen that once in our codebase :), check if you correctly unpack normalmap normals, by tex2D(..)*2-1...
Toji
Toji
HA HA! Got it!



It was a really obvious problem in retrospect too. Pop quiz time!

If the following works in HLSL:

float3x3 TBNMatrix( Tangent, Bitangent, Normal );

why won't the following work in GLSL?

mat3 TBNMatrix( Tangent, Bitangent, Normal );

I'll wait...
...
...
..
..
.
.

Because (and I'm kicking myself now because of this) Matricies are transposed in OpenGL! (Duh!) So of course my old shader code wouldn't work, cause the rotation matrix was flipped! (Stupid stupid stupid!)

So, the final, correct vertex shader (fragment shader didn't change) is:

varying vec3 lightVec;attribute vec3 Tangent, Bitangent;uniform vec4 LightPos;uniform mat4 WorldMatrix;uniform mat4 InvWorldMatrix;   void main(){   //Pass out our transformed Position and TexCoords   gl_Position = ftransform();   gl_TexCoord[0] = gl_MultiTexCoord0;      //Calculate the Tangent-Space matrix. This may need tweaking   mat3 TBNMatrix = mat3(Tangent.x, Bitangent.x, gl_Normal.x,						 Tangent.y, Bitangent.y, gl_Normal.y,						 Tangent.z, Bitangent.z, gl_Normal.z);      //Get the "world position" of the vertex   vec4 vectWorld = WorldMatrix * gl_Vertex;   //Cast a ray to the light from that position   vec4 lightToPoint = LightPos - vectWorld;      //Transform the light-ray to world space   vec3 lightWorld = (InvWorldMatrix * lightToPoint).xyz;      //Put the light direction in tangent space   lightVec = TBNMatrix * lightWorld;  }


Thank you EVERYONE so much for your helpful suggestions! I never would have gotten it without you! Any of you who I haven't rated up yet, it's coming!

Thanks again!
-Toji
// The user formerly known as Tojiro67445, formerly known as Toji [smile]
Code-R
Code-R
I'm just curious, and mostly wrong, but won't changing
lightVec = TBNMatrix * lightWorld;


to

lightVec = lightWorld * TBNMatrix;


work, instead of transposing the matrix?

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.