# Normal mapping issues (solved)

This topic is 1884 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

Ok, I have been trying to do some normal mapping, and I'm getting some strange behavior. I'm sure I have some misconceptions that are complicating the issue so I wanted to ask some questions. I am using, or rather trying to use, the technique described on this blog. I had used its predecessor in my work with DX9 that can be found in the ShaderX5 book without issues. Here is my HLSL code for computing the 'cotangent frame' in case I made a mistake converting from GLSL.

float3x3 cotangent_frame( float3 N, float3 p, float2 uv )
{
// get edge vectors of the pixel triangle
float3 dp1 = ddx( p );
float3 dp2 = ddy( p );
float2 duv1 = ddx( uv );
float2 duv2 = ddy( uv );

// solve the linear system
float3 dp2perp = cross( dp2, N );
float3 dp1perp = cross( N, dp1 );
float3 T = dp2perp * duv1.x + dp1perp * duv2.x;
float3 B = dp2perp * duv1.y + dp1perp * duv2.y;

// construct a scale-invariant frame
float invmax = rsqrt( max( dot(T,T), dot(B,B) ) );
return float3x3( T * invmax, B * invmax, N );
}



What I am trying to do is etch a river system onto a hex game board. If you look up some of my previous posts you can see what I have been working on the past year or so. Here is an example normal and height map for a river piece:

[attachment=18806:LargeFarMedLarge.png][attachment=18807:LargeFarMedLargeN.png]

The normal map was made with a GIMP plugin, the color scheme has the up vector at r127,g127,b255. That is the color of most of the normal map and also at the river bottom. I am assuming, possibly incorrectly, that this value should leave the interpolated normal alone unperturbed.

Here is a description of the strange behavior. When I use the following shader code to perturb the normal, I get something that looks correct on one of my machines, but not the other. I am sampling the height map first (for multiple reasons, such as later coloring the river bottom with a sandy texture or whatever) then testing that data to see if there is river data at that pixel. Then I perturb the normal.

		if(rivColor.r > 0.0)
{
rivTan = cotangent_frame(input.Normal,-input.View,input.RiverTexCoord);
rivNorm = riverNormals.Sample(samLinear, input.RiverTexCoord).xyz;
normal = normalize(mul(rivNorm * 2.0f - 1.0f,rivTan));
}


On the other gpu, the river bed is surrounded by an ugly black line. In any case, the following picture is actually what I want to see, and is from the gpu without the ugly line.

[attachment=18803:Rivers1.jpg]

Here is what I don't understand. If my assumption is correct, and the normal map value r127, g127, b255 does not perturb the normal, I shouldn't have to test for height data for the normal map to look right. However, when I comment out the condition, It completely changes the results. Not only are the formerly hidden parts wrong, but even the part that wasn't masked away is completely changed as well.

		//if(rivColor.r > 0.0)
//{
rivTan = cotangent_frame(input.Normal,-input.View,input.RiverTexCoord);
rivNorm = riverNormals.Sample(samLinear, input.RiverTexCoord).xyz;
normal = normalize(mul(rivNorm * 2.0f - 1.0f,rivTan));
//}



Note that even the rivers no longer give the proper normals like they did in the above example.

[attachment=18804:Rivers2.jpg]

Can anyone explain to me why this is happening? I must be doing something wrong. I might not care if I got reliable results from the first example, but it only works on one of my gpus for some reason. I don't understand why one example seems to work and the other doesn't.

Edited by cephalo

##### Share on other sites

Ok, I'm getting a little bit closer with the following code changes. First, I transposed the cotangent frame:

float3x3 cotangent_frame( float3 N, float3 p, float2 uv )
{
// get edge vectors of the pixel triangle
float3 dp1 = ddx( p );
float3 dp2 = ddy( p );
float2 duv1 = ddx( uv );
float2 duv2 = ddy( uv );

// solve the linear system
float3 dp2perp = cross( dp2, N );
float3 dp1perp = cross( N, dp1 );
float3 T = dp2perp * duv1.x + dp1perp * duv2.x;
float3 B = dp2perp * duv1.y + dp1perp * duv2.y;

// construct a scale-invariant frame
float invmax = rsqrt( max( dot(T,T), dot(B,B) ) );
//return float3x3( T * invmax, B * invmax, N );
return transpose(float3x3( T * invmax, B * invmax, N ));
}



Then, I tried to remove any sampling issues by merely setting the sample to (0,0,1) which should not perturb the normal.

		//if(rivColor.r > 0.0)
//{
rivTan = cotangent_frame(input.Normal,input.View,input.RiverTexCoord);
rivNorm = riverNormals.Sample(samLinear, input.RiverTexCoord).xyz;
rivNorm = normalize(rivNorm * 2.0f - 1.0f);
//rivNorm.y = -rivNorm.y;
rivNorm = normalize(float3(0,0,1));
normal = normalize(mul(rivTan, rivNorm));
//}



This gave the expected result that the normals were unperturbed! I can change the normal in order to perturb it manually and that works also. However, trying to use the sampled data is still giving garbage results. Since I am a VS Express user, I can't actually see the value of rivNorm directly.

I would assume that a value of 127,127,255 would sample as (0.5f, 0.5f. 1.0f) and then *2 - 1.0f would give (0,0,1) but that's not happening.

##### Share on other sites

Ok, so I figured it out. Not only do I have to transpose the cotangent frame, but I also had two texture arrays, one for the height map and one for the normal map, and had also failed to specify which register they should go in. I decided to use my normal texture as a color texture to see what it would look like, and was shocked to see my height map instead! Using the conditional statement in the pixel shader worked sortof because it somehow forced the compiler to use the right texture. Setting the t registers to their appropriate slots fixed most of my problems, as I was sampling the wrong texture.

The only problem I have left is that using that conditional statement, which I suppose I don't really need, still looks bad on my nVidia gpu.

[attachment=18839:Rivers3.jpg]

If anybody knows what automatic process is mucking this up, I would love to hear about it. In any case, it appears that my normal mapping code is correct now.

##### Share on other sites

To "answer" your PM request: Sorry cephalo, I couldn.t make much sense of the problem other than suspecting a driver issue on the non-working GPU.

Now you have beaten me by mere seconds :)

For the newest development, too I can only offer a shot in the dark : Try translating the if into a lerp. I think you should rather blend different parts than if/else, sort of texture splatting (remember the barycentric voronoi thread we had).

##### Share on other sites

Thanks unbird as always! I have another problem coming up regarding my displacement mapping. It looks astonishingly bad. I'll start another thread when I'm ready. Stay tuned!

• ### What is your GameDev Story?

In 2019 we are celebrating 20 years of GameDev.net! Share your GameDev Story with us.

• 9
• 13
• 9
• 9
• 15
• ### Forum Statistics

• Total Topics
634071
• Total Posts
3015340
×