Jump to content
  • Advertisement
Sign in to follow this  

Normal Mapping / DX compute tangent problem!

This topic is 3568 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi there, im having some very strange problems with my normal mapping code!. Im pretty sure it was working a few weeks ago and i havent messed with the graphical side of my project, although i could be wrong about it working properly! I may just not have noticed! Additionally i have ths same shader code running in an older demo, and after 20 mins of wathcin it im CONVINVED it works fine in the old demo! Basically problem 1 is featured in the screenshot below. The Problem is that the lighting on the normal map appears to be somewhat offset from the direction of the actual light. This goes so far as, if the light is orbiting around workd Y in a clockwise motion, i SWEAR that the normal map lighting appears to be changinegin a counter-clockwise motions! (as if the light were orbiting the opposite way!). http://img89.imageshack.us/my.php?image=nmapprobiz0.jpg The second problem may just be an extreme variation on the first. Screenshot below again. Essentially (far more rarely though) lighting on a piece of geometry appears to be back to front!. The side that should be lit will be in shadow, whilst the OPPOSITE (e.g. rear) side appears lit as a light passes by the front. http://img225.imageshack.us/my.php?image=normmapprob2vg3.jpg My friend seems to think that my problem lies with using DX to generate my tangents (im currently using D3DXComputeTangentFrameEx()). He claims he had similar problems that literally vanished the second he replaced the DX function with his own tangent generation function. According to him DXComputeTangent function are sent from hell by satan himself to annoy and vex programmers with confusing strange, often hard to reproduce, 'only-just-not-quite-right-so-you-can't-quite-put-your-finger-on-it' errors. So i think hes telling me to ditch it. However if it isnt a problem with that i could spend a while writing tangent generation code to at best, end up where i am now, or waste even more time thinking that my tangent generation code isnt right when its the original error still and i cant be sure!. Hmm anyone have any thoughts?? (or similiar experiences) Either way SOMETHING is seriously wierd ! And i cant for the life of me figure out what and its driving me nuts! Its so frustrating since ive had many normal mapped demos in the past, seemingly without problems! Ill put my shader code here too just in case there is something im doing thats really stupid! (ps i know its heavily messy and non optimized at this point!)

// define the light types used (same enums as CLightTemplate)
#define kPoint 0
#define kDirectional  1
#define kSpot  2

// Max num lights. Keep this synchd to CRenderManager MAX_NUM_LIGHTS
#define MAX_NUM_LIGHTS 5

int gActualNumLights; //pass in actual number of lights, for for loop checks

int	   gLightType[MAX_NUM_LIGHTS];
float3 gLightPosition[MAX_NUM_LIGHTS]; 
float3 gLightColor[MAX_NUM_LIGHTS];   
float3 gLightDirection[MAX_NUM_LIGHTS];
float  gLightIntensity[MAX_NUM_LIGHTS];
float  gLightCosHalfAngle[MAX_NUM_LIGHTS];

uniform extern float4x4 gWorld; 
uniform extern float4x4 gVP;
uniform extern float4x4 gWorldInverseTranspose; //used to transform normal without shearing
uniform extern float4 gAmbientMaterial; 
uniform extern float4 gAmbientLight;
uniform extern float4 gDiffuseMaterial;
uniform extern float4 gSpecularMaterial; 
uniform extern float gSpecularPower;
uniform extern float3 gEyePosW;

uniform extern texture gTex;
uniform extern texture gNormalMapTex;

sampler Tex0S = sampler_state
	Texture = <gTex>;
	MinFilter = LINEAR;
	MagFilter = LINEAR;
	MipFilter = LINEAR;
	AddressU  = WRAP;
	AddressV  = WRAP;

sampler NormMapS = sampler_state
	Texture = <gNormalMapTex>;
	MinFilter = LINEAR;
	MagFilter = LINEAR;
	MipFilter = LINEAR;
	AddressU  = WRAP;
	AddressV  = WRAP;

struct OutputVS
    float4 posH     : POSITION0;
    float3 normalL  : TEXCOORD0;
    float3 posW     : TEXCOORD1;
    float2 tex      : TEXCOORD2; 
    float3 tangentL : TEXCOORD3;

OutputVS TransformVS(float3 posL : POSITION0, float3 normalL : NORMAL0, float3 tangentL : TANGENT0, float3 tex0 : TEXCOORD0)
        // Zero out our output.
	OutputVS outVS = (OutputVS)0;
	//pass on normal and tangent
	outVS.normalL   = normalL;
	outVS.tangentL  = tangentL;  
	//Transform vertex position to world space(easier for spec lighting)
	float3 posW = mul(float4(posL,1.0f), gWorld).xyz;
	outVS.posW = posW;
	// Transform to homogeneous clip space.
	outVS.posH = mul(float4(posW, 1.0f), gVP);
	outVS.tex     = tex0;
	// Done--return the output.
       return outVS;

float4 TransformPS( float3 normalL  : TEXCOORD0,
					float3 posW     : TEXCOORD1,
					float2 tex      : TEXCOORD2, 
					float3 tangentL : TEXCOORD3 ) : COLOR
	// Normal Map Extraction
	// Step 1) renormalise (interpolated)
	normalL  = normalize(normalL);
	tangentL = normalize(tangentL);
	// Step 2) Calculate biTangent
	float3 biTangentL = normalize(cross( normalL, tangentL ));
	float3x3 InvTangentMatrix = float3x3(tangentL, biTangentL, normalL);
	// Step 3) Extract normal in correct range (stored 0 -> 1 convert to -1 ->1 
	//float3 TextureNormal = (tex2D( NormMapS, tex )*tex2D( NormMapS, tex ));
	float3 TextureNormal = 2.0f *  tex2D( NormMapS,tex ) - 1.0f; // Scale from 0->1 to -1->1
	// step 4) now transform from tangentspace -> model space -> world space
	float3 normalW = normalize( mul( mul( TextureNormal, InvTangentMatrix ), gWorld) );
	//float4 FinalColor = float4((normalW.x*0.5f)+0.5f,(normalW.y*0.5f)+0.5f,(normalW.z*0.5f)+0.5f,1.0f);
	//return FinalColor;

	// Do one off calculations
	// Calculate the vector to the eye
	float3 toEye = normalize(gEyePosW - posW);
	// Now do PER LIGHT calculations
	float3 toLight;
	float3 reflected;
	float3 halfway;
	float3 TotalSpecular = 0.0f;
	float3 TotalDiffuse = 0.0f; //add ambient on here for now? 
	float  diffIntensity;
	float  specIntensity;
	float  dist;
	for(unsigned int i = 0; i < gActualNumLights; i++)
		if(gLightType == kPoint || gLightType == kSpot)
			// Now calculate the to light vector
			toLight = normalize(gLightPosition - posW);					
			// Directional lights specify their direction globally
			toLight = -normalize(gLightDirection); 			
		// Compute the reflection vector
		reflected = reflect(-toLight, normalW);
		// proceed exactly in same way (for now) for pixels in point lights and pixels in spot light cones
		if(gLightType == kPoint || (gLightType == kSpot && dot(gLightDirection,-toLight) > gLightCosHalfAngle) || gLightType == kDirectional )
			// Next calculate the diffuse light intensity hitting the vertex
			diffIntensity = saturate(dot(toLight, normalW));
			// Attenuation
			if(gLightType == kDirectional)
			dist = 1;
			dist = distance(gLightPosition, posW);
			halfway = normalize( toEye + toLight );
			// Use dot to work out intensity of specular (how much actually hits eye)
			// the dot of the reflected and to eye vectors, can be -1 -> 1 so we use
			// max(x,0.0f) to make sure we get a dot in range 0 ->1, then we raise
			// result by the specular power
			specIntensity = pow(saturate(dot(halfway,normalW)), gSpecularPower);
			// Now calculate all the light of each type separately
			TotalSpecular += specIntensity  * (gLightIntensity * (gSpecularMaterial*gLightColor)) /dist;	
			// Calculate diffuse and ambient, and add together
			// Standard equation: Diffuse = max(0, N.L)
			TotalDiffuse  +=  diffIntensity * (gLightIntensity*(gDiffuseMaterial*gLightColor))/dist;
	// calculate ambient
	float3 ambient = (gAmbientMaterial*gAmbientLight).rgb;

	// Now calculate final color based on Total specular/diffuse + ambient
	float4 texColor = tex2D(Tex0S, tex);
	float4 finalColor;
	finalColor.rgb = (TotalDiffuse * texColor  + (TotalSpecular * texColor.a) ); //tweak max attenuation so we dont get /0.01 etc n get super bright
	finalColor.rgb += ambient * texColor ;
	//finalColor.rgb = dot(finalColor, float3(0.33f, 0.33f, 0.33f));
	finalColor.a = texColor.a; //change to texcolor.a for spec map

	return finalColor;


technique NormalMapTech
    pass P0
        // Specify the vertex and pixel shader associated with this pass.
        vertexShader = compile vs_3_0 TransformVS();
        pixelShader  = compile ps_3_0 TransformPS();


Share this post

Link to post
Share on other sites
In case your UVs are flipped than this won't work correctly because you calculate the binormal by the cross product of normal and tangent (ideally your binormal should be provided by the model). Try flipping the UVs by the X,Y or both axis.
You should also try rewriting the tangent calculation code, no point in trying to solve the problem just because you "dont want to waste time" on a tangent maker.
tangent code in case you don't have it

Share this post

Link to post
Share on other sites
well i implemented my own tangent generation function as advised.

Seems the strange errors are still present however :S.

this is driving me crazy!!

I tried flipping the UVs too but couldnt get it to look right with any combination so i dont think its that( will keep trying in case i missed something )

Anybody got any other suggestions, this is absolutely driving me NUTS.

Share this post

Link to post
Share on other sites
hmm ok.. well i set the incoming tex UVs to have the existing U coord , but 0.0f - V for the V coord... and normal mapping SEEMS to be ok on my cube now. (however in other places it still isnt ! argh! this is a pain to fix!)

Share this post

Link to post
Share on other sites
Why not have your modelling package export the tangents and bitangents, and just use them?

Some of your faces might have mirrored UVs which will cause the tangent/bitangent to be pointing in the wrong direction for that face.

Share this post

Link to post
Share on other sites
I would love to if i could, but im stuck in a situation where i only have a handful of premade models for my project :S. Its really irritating!. I cant model / dont really have access to a decent modelling package as it stands, so im stuck generating them myself which is pretty irritating.

I checked an old backup copy of my project from a few weeks back and im damned sure normal mapping is working fine in that copy on the same models.

Ive checked my current shader against the old one and it seems near identical. Im very confused!

Share this post

Link to post
Share on other sites
Here is my code for normal mapping (I've stripped it down so you'll only see the lighting) (It's dx10 but its 99% identical to dx9)

PS_IN VS(VS_IN input)
PS_IN output=(PS_IN)0;

float4 wp=mul(float4(input.pos,1.0f),world);


float3 tan=input.tangent;
float3 binorm=input.binormal;
float3 norm=input.normal;

float3x3 tsm;



return output;

float4 PS(PS_IN input) :SV_TARGET

float3 light=normalize(input.light);

float3 normal=2*normalTexture.Sample(sampler1,input.texcoord)-1.0f;

float dp=saturate(dot(light,normal));

return dp;

I can also provide you with some simple test models if you want

Share this post

Link to post
Share on other sites
Original post by NovaBlack
Ive checked my current shader against the old one and it seems near identical. Im very confused!

Ok, so what exactly is different?

Share this post

Link to post
Share on other sites
The only thing different in my old code is that i didnt have support for directional lights.. so its just one statement missing from the main 'if' check.

Hmm ive just created a completely new environment with my editor, using the new normal map shader (where tex.V is flipped to 0.0f - tex.V) and everything looks ... fine :S

Which is odd.... Im not saving any level data to do with normal mapping whatsoever :S.. so there shouldnt really be any difference between what works and what doesnt.. but this new environment seems to look ok (with the oddly ammended shader).

Ill try rebuilding the original environment from scratch (should take a few hours) and ill see if that helps. Perhaps i scaled some geometry inside out or something and caused some strange problems, im not entirely sure. According to the tests ive just done on a new environment, it seems that it is (at least for this instance) working ok... so ill keep my fingers crossed and rebuild.

Cheers for the code nexus. As far as i can see i have definitely got the concept behind what im doing correct. As was said earlier it probably does boil down to the face im not using the binormal directly from the model itself, and some issue arises whereby the cross product result to gen the binormal isnt what i expect.

Ill give an update tomorrow and see if its sorted. Cheers for all the help (AND PATIENCE!) so far guys!

Share this post

Link to post
Share on other sites
Sign in to follow this  

  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!