Sign in to follow this  
dustrat

Normal mapping banding problem

Recommended Posts

Hi! I've been writing a vertex and a pixel shader for normal mapping which uses the phong lighting model. I have some serious banding problems as shown in the following screenshot http://users.du.se/~d98ped/banding.jpg The screenshot is showing the colormap in the lower left corner and the normal map to the right of the colormap. The shaders seem to work perfectly in RenderMonkey, but when I use them in the game engine, the banding artifacts appear. FYI the game engine is RenderWare Graphics 3.7 and the underlying 3D API is DirectX 9. Does anyone have any clue as to what can cause this problem? Vertex shader:
float4x4 view_proj_matrix;
float4x4 inv_view_matrix;
float bumpiness;

float4 light_position;
float4 eye_position;

struct VS_INPUT_STRUCT
{
   float4 position:     POSITION;
   float3 normal:       NORMAL;
   float2 texcoord0:    TEXCOORD0;
   float3 binormal:     BINORMAL0;
   float3 tangent:      TANGENT0;
};

struct VS_OUTPUT_STRUCT
{
   float4 position:     POSITION;
   float2 bump_map:     TEXCOORD0;
   float3 light_vector: TEXCOORD1;
   float3 half_angle:   TEXCOORD2;
};

VS_OUTPUT_STRUCT main( VS_INPUT_STRUCT input )
{
   VS_OUTPUT_STRUCT output;

   output.position = mul(view_proj_matrix, input.position);
   output.bump_map = input.texcoord0;
   
   // Converts from object space to tangent space
   float3x3 matObjToTangent = {input.tangent * bumpiness, input.binormal * bumpiness, input.normal};
   
   // light vector from world space -> object space -> tangent space
   float3 temp_light_position = mul(inv_view_matrix, light_position);
   float3 temp_light_vector2 = normalize(temp_light_position - input.position);
   output.light_vector = mul(matObjToTangent, temp_light_vector2);
   
   // view vector from world space -> object space -> tangent space
   float3 temp_eye_position = mul(inv_view_matrix, eye_position);
   float3 temp_view_vector2 = normalize(temp_eye_position - input.position);
   float3 temp_view_vector = mul(matObjToTangent, temp_view_vector2);

   output.half_angle = normalize(temp_view_vector + output.light_vector);

   return output;
}

Pixel Shader:
//
// Light constants
//
float4 light_color;
float light_intensity;

//
// Material constants
//
float4 specular;
float4 ambient;
float4 diffuse;
float specular_power;

//
// Textures
//
sampler2D base_map;
sampler2D bump_map;

struct PS_INPUT_STRUCT
{
   float2 bump_map:     TEXCOORD0;
   float3 light_vector: TEXCOORD1;
   float3 half_angle:   TEXCOORD2;
};

// Blinn/Phong lighting model
float4 Phong(float2 dots : POSITION) : COLOR
{
   float NdotL = dots.x;
   float NdotH = dots.y;
   float diffuse_reflection = max(NdotL, 0.0);
   float specular_reflection = pow(NdotH, specular_power);
   
   if (NdotL <= 0) 
      specular_reflection = 0;
   
   return float4(diffuse_reflection, diffuse_reflection, diffuse_reflection, specular_reflection);
}

float4 main( PS_INPUT_STRUCT input ) : COLOR
{
   float4 colorMap = tex2D(base_map, input.bump_map);
   float3 N = tex2D(bump_map, input.bump_map) * 2.0 - 1.0;
   
   float NdotL = dot(N, input.light_vector);
   float NdotH = dot(N, input.half_angle);
   float4 light = Phong(float2(NdotL, NdotH)) * light_intensity;
   
   return ambient * colorMap + light * diffuse * colorMap + light.a * specular;

Share this post


Link to post
Share on other sites
Quote:

ou are talking about the cup thing in the picutre? I'm just guessing, but possibly your tangent/binormal vectors have not been (pre)calculated properlly.


I believe this can be the case. Is there any way to calculate those normals in the vertex shader? Can I fetch other vertices somehow?

Share this post


Link to post
Share on other sites
Quote:
No. You have to calculate at least on of them outside of vertex shader. Computing tangent space basis vectors for an arbitrary mesh by Eric Lengyel.


Good thing that I own that book. Do you think that it is the source to the problem, by the way?

Edit: fixed some typos

Share this post


Link to post
Share on other sites
That problem is not banding, for sure. In banding, the values are basically right, there's just steps in them. In your image, the values aren't even right.

You can construct the tangent in the vertex shader by crossing the normal with the binormal.

You can pass the binormal using any available vertex attribute. For example, if you're only using one texture coordinate channel, pass the binormal in the second texture channel. If you're using two texture coordinate channels, pass the binormal in the third. If you're using GLSL, they've gone to the point where you can just number the data input channels 0-15, and it's up to you to determine which ones are used and do what.

Share this post


Link to post
Share on other sites
Dustrat:
You would send them as uniform variables or as texture-coordinates...

And now a question in a whole other department... how on earth can a guy like you have RenderWare and not knowing how to send stuff to a shader? FYI: I'm not trying to flame you, just curious! :)

Best regards
Roquqkie

oh and btw. It's called BiTANGENT!!!! Not BiNormal!!!! ;)

edit: Damn, hPlus got there before me! >:)

Share this post


Link to post
Share on other sites
Quote:
Original post by Roquqkie
Dustrat:
You would send them as uniform variables or as texture-coordinates...

And now a question in a whole other department... how on earth can a guy like you have RenderWare and not knowing how to send stuff to a shader? FYI: I'm not trying to flame you, just curious! :)

Best regards
Roquqkie

oh and btw. It's called BiTANGENT!!!! Not BiNormal!!!! ;)

edit: Damn, hPlus got there before me! >:)


Well, the thing is Im learning how to use shaders with renderware on my own...there is no documentation at all for a lot of stuff...so many times its trial and error all the way...

Share this post


Link to post
Share on other sites
Ah okay ;)

You should download a shader-editor, that way you can skip much of the trial-and-error process (by knowing that your shader looks correct in the first place). I really like the tools from NVidia, so go there and pick one up: www.nvidia.com/developer/

If you have any questions, feel free to mail me.

Best regards,
Roquqkie
mikkel@cgrender.dk

Share this post


Link to post
Share on other sites
Quote:
Original post by Roquqkie
Ah okay ;)

You should download a shader-editor, that way you can skip much of the trial-and-error process (by knowing that your shader looks correct in the first place). I really like the tools from NVidia, so go there and pick one up: www.nvidia.com/developer/

If you have any questions, feel free to mail me.

Best regards,
Roquqkie
mikkel@cgrender.dk


The main problem does not lie in creating the shaders I believe. I'm getting fairly used to doing things in rendermonkey by now. The trouble is that I have to create my own custom pipelines in RenderWare in order to use my own shaders. That means I must write vertex shaders that performs skinning and so on..

Share this post


Link to post
Share on other sites
I have solved it!

just two lines of codes were required:
RpD3D9GeometrySetUsageFlags(pGeometry, rpD3D9GEOMETRYUSAGE_CREATETANGENTS);

and the additional cross product of the tangent and the normal in the vertex shader!

Share this post


Link to post
Share on other sites
Hi dustrat.

I am right now in the situation you where a while back. I am using renderware and trying to implement shaders, but there is absolutely no documentation at all to be found. I am hoping you can help me with a few questions.

1. How did you get the camera view matrix from renderware? Did you have to write your own convertion function from RwMatrix to D3DXMATRIX?

2. I cnat find any support for HLSL in renderware. Did you call D3D9 functions directly or did you write the shaders in assembly?

3. How did you implement your shaders in the renderware pipeline?

I'd appreciate any help I can get on this. I've got about 1 week to get this into a game and it turned out much more complicated than I thought. Integrating the shader into renderware is where the whjole problem lies.

Tack för hjälpen

Jakob

Share this post


Link to post
Share on other sites
Quote:
Original post by tealrabbit
1. How did you get the camera view matrix from renderware? Did you have to write your own convertion function from RwMatrix to D3DXMATRIX?

2. I cnat find any support for HLSL in renderware. Did you call D3D9 functions directly or did you write the shaders in assembly?

3. How did you implement your shaders in the renderware pipeline?

I'd appreciate any help I can get on this. I've got about 1 week to get this into a game and it turned out much more complicated than I thought. Integrating the shader into renderware is where the whjole problem lies.

Tack för hjälpen

Jakob


Hello Jakob, sorry for the late reply. I havent checked gamedev for a while, and for some reason I dont get any mails when I get replies.

1. Converting between matrices isn't too hard. Here is an example:

D3DXMATRIX m1;
RwMatrix* m2 = RwMatrixCreate();

m1.m[0][0] = m2->right.x;
m1.m[0][1] = m2->right.y;
m1.m[0][2] = m2->right.z;
m1.m[0][3] = 0.0f;

m1.m[1][0] = m2->up.x;
m1.m[1][1] = m2->up.y;
m1.m[1][2] = m2->up.z;
m1.m[1][3] = 0.0f;

m1.m[2][0] = m2->at.x;
m1.m[2][1] = m2->at.y;
m1.m[2][2] = m2->at.z;
m1.m[2][3] = 0.0f;

m1.m[3][0] = m2->pos.x;
m1.m[3][1] = m2->pos.y;
m1.m[3][2] = m2->pos.z;
m1.m[3][3] = 1.0f;

;

You might need to transpose the matrix too.

2. If you are using the d3d9 version of RenderWare, you have some platformdependent functions prefixed RwD3D9 that you must use in order to make shaders work. Look in the documentation. My experience is that I must compile asm-shader code to "tokens" in order to use them in Rw, and then create custom pipelines with overloaded render callbacks.

I'm using ATI's RenderMonkey to create vertex and pixel shaders. From RenderMonkey, I get the shaders in ASM-format and after that I use the microsoft supplied programs vsa and psa to assemble the code into tokens. Take a look at the renderware example "vshader".

3. The vshader example also shows how to create a custom pipeline.

Lycka Till!

/Dustrat

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this