Normal mapping banding problem

Started by
12 comments, last by dustrat 19 years, 4 months ago
Hi! I've been writing a vertex and a pixel shader for normal mapping which uses the phong lighting model. I have some serious banding problems as shown in the following screenshot http://users.du.se/~d98ped/banding.jpg The screenshot is showing the colormap in the lower left corner and the normal map to the right of the colormap. The shaders seem to work perfectly in RenderMonkey, but when I use them in the game engine, the banding artifacts appear. FYI the game engine is RenderWare Graphics 3.7 and the underlying 3D API is DirectX 9. Does anyone have any clue as to what can cause this problem? Vertex shader:

float4x4 view_proj_matrix;
float4x4 inv_view_matrix;
float bumpiness;

float4 light_position;
float4 eye_position;

struct VS_INPUT_STRUCT
{
   float4 position:     POSITION;
   float3 normal:       NORMAL;
   float2 texcoord0:    TEXCOORD0;
   float3 binormal:     BINORMAL0;
   float3 tangent:      TANGENT0;
};

struct VS_OUTPUT_STRUCT
{
   float4 position:     POSITION;
   float2 bump_map:     TEXCOORD0;
   float3 light_vector: TEXCOORD1;
   float3 half_angle:   TEXCOORD2;
};

VS_OUTPUT_STRUCT main( VS_INPUT_STRUCT input )
{
   VS_OUTPUT_STRUCT output;

   output.position = mul(view_proj_matrix, input.position);
   output.bump_map = input.texcoord0;
   
   // Converts from object space to tangent space
   float3x3 matObjToTangent = {input.tangent * bumpiness, input.binormal * bumpiness, input.normal};
   
   // light vector from world space -> object space -> tangent space
   float3 temp_light_position = mul(inv_view_matrix, light_position);
   float3 temp_light_vector2 = normalize(temp_light_position - input.position);
   output.light_vector = mul(matObjToTangent, temp_light_vector2);
   
   // view vector from world space -> object space -> tangent space
   float3 temp_eye_position = mul(inv_view_matrix, eye_position);
   float3 temp_view_vector2 = normalize(temp_eye_position - input.position);
   float3 temp_view_vector = mul(matObjToTangent, temp_view_vector2);

   output.half_angle = normalize(temp_view_vector + output.light_vector);

   return output;
}

Pixel Shader:

//
// Light constants
//
float4 light_color;
float light_intensity;

//
// Material constants
//
float4 specular;
float4 ambient;
float4 diffuse;
float specular_power;

//
// Textures
//
sampler2D base_map;
sampler2D bump_map;

struct PS_INPUT_STRUCT
{
   float2 bump_map:     TEXCOORD0;
   float3 light_vector: TEXCOORD1;
   float3 half_angle:   TEXCOORD2;
};

// Blinn/Phong lighting model
float4 Phong(float2 dots : POSITION) : COLOR
{
   float NdotL = dots.x;
   float NdotH = dots.y;
   float diffuse_reflection = max(NdotL, 0.0);
   float specular_reflection = pow(NdotH, specular_power);
   
   if (NdotL <= 0) 
      specular_reflection = 0;
   
   return float4(diffuse_reflection, diffuse_reflection, diffuse_reflection, specular_reflection);
}

float4 main( PS_INPUT_STRUCT input ) : COLOR
{
   float4 colorMap = tex2D(base_map, input.bump_map);
   float3 N = tex2D(bump_map, input.bump_map) * 2.0 - 1.0;
   
   float NdotL = dot(N, input.light_vector);
   float NdotH = dot(N, input.half_angle);
   float4 light = Phong(float2(NdotL, NdotH)) * light_intensity;
   
   return ambient * colorMap + light * diffuse * colorMap + light.a * specular;

Advertisement
You are talking about the cup thing in the picutre? I'm just guessing, but possibly your tangent/binormal vectors have not been (pre)calculated properlly.

- Thomas Cowellwebsite | journal | engine video

Quote:
ou are talking about the cup thing in the picutre? I'm just guessing, but possibly your tangent/binormal vectors have not been (pre)calculated properlly.


I believe this can be the case. Is there any way to calculate those normals in the vertex shader? Can I fetch other vertices somehow?
Quote:Original post by dustrat
Is there any way to calculate those normals in the vertex shader? Can I fetch other vertices somehow?

No. You have to calculate at least one of them outside of vertex shader. Computing tangent space basis vectors for an arbitrary mesh by Eric Lengyel.
You should never let your fears become the boundaries of your dreams.
Quote:No. You have to calculate at least on of them outside of vertex shader. Computing tangent space basis vectors for an arbitrary mesh by Eric Lengyel.


Good thing that I own that book. Do you think that it is the source to the problem, by the way?

Edit: fixed some typos
How would I pass the calculated bitangent and tangent vectors to the vertex shader after I have calculated them?
That problem is not banding, for sure. In banding, the values are basically right, there's just steps in them. In your image, the values aren't even right.

You can construct the tangent in the vertex shader by crossing the normal with the binormal.

You can pass the binormal using any available vertex attribute. For example, if you're only using one texture coordinate channel, pass the binormal in the second texture channel. If you're using two texture coordinate channels, pass the binormal in the third. If you're using GLSL, they've gone to the point where you can just number the data input channels 0-15, and it's up to you to determine which ones are used and do what.
enum Bool { True, False, FileNotFound };
Dustrat:
You would send them as uniform variables or as texture-coordinates...

And now a question in a whole other department... how on earth can a guy like you have RenderWare and not knowing how to send stuff to a shader? FYI: I'm not trying to flame you, just curious! :)

Best regards
Roquqkie

oh and btw. It's called BiTANGENT!!!! Not BiNormal!!!! ;)

edit: Damn, hPlus got there before me! >:)
Quote:Original post by Roquqkie
Dustrat:
You would send them as uniform variables or as texture-coordinates...

And now a question in a whole other department... how on earth can a guy like you have RenderWare and not knowing how to send stuff to a shader? FYI: I'm not trying to flame you, just curious! :)

Best regards
Roquqkie

oh and btw. It's called BiTANGENT!!!! Not BiNormal!!!! ;)

edit: Damn, hPlus got there before me! >:)


Well, the thing is Im learning how to use shaders with renderware on my own...there is no documentation at all for a lot of stuff...so many times its trial and error all the way...
Ah okay ;)

You should download a shader-editor, that way you can skip much of the trial-and-error process (by knowing that your shader looks correct in the first place). I really like the tools from NVidia, so go there and pick one up: www.nvidia.com/developer/

If you have any questions, feel free to mail me.

Best regards,
Roquqkie
mikkel@cgrender.dk

This topic is closed to new replies.

Advertisement