Toon Shader Problem

Started by
10 comments, last by bfogerty 15 years, 8 months ago
Hello Everyone. I am trying to write a toon shader using Render Monkey. The basic approach that I am taking is to get the dot product between the light direction and the normal vector of the incoming vertex. I will then use that value as the u texture coordinate for my toon texture. The following is my source,

float4x4 matViewProjection;
float4 fLightPos;

struct VS_INPUT 
{
   float4 Position : POSITION0;
   float4 Normal : NORMAL0;
   float4 TexCoord : TEXCOORD0;
   
};

struct VS_OUTPUT 
{
   float4 Position : POSITION0;
   float4 TexCoord : TEXCOORD0;
   
};

VS_OUTPUT vs_main( VS_INPUT Input )
{
   VS_OUTPUT Output;

   Output.Position = mul( Input.Position, matViewProjection);
   Output.TexCoord = Input.TexCoord;
   
   fLightPos = float4(0.3,0.3,0.3,1);
   float4 lightDir = Input.Position - fLightPos;
   lightDir.w = 0;
   Input.Normal.w = 0;
   lightDir = mul(lightDir, matViewProjection);
   Input.Normal = mul(Input.Normal, matViewProjection);
   
   float x = dot(lightDir,Input.Normal);
   
   if(x<0)
   {
      x = 0;
   }
   
   //Output.TexCoord = Input.TexCoord;
   Output.TexCoord.x = x;
   Output.TexCoord.y = 0.5f;
   
   return( Output );
   
}
For some reason, my mesh's texture is always dark, meaning that x seems to always be staying around 0. I am not sure why. Debugging is a bit of a pain and I am wondering if anyone seems a mistake in my logic. Thanks!
Advertisement
A dot product will give you results in the range of [-1,1], whereas your texture coords should be in the range [0,1]. I see you've got a <0 check, but instead shouldn't you remove that and just convert from one range to the other?
Don't use if:
float x = max(dot(lightDir, Input.Normal), 0);


Also, don't do the dot product in perspective space, that is, don't multiply by matViewProjection, only by matView. Or mat. Or neither.
Thanks, I updated my code to the following but I still get the same effect. Any other ideas?
float4x4 matViewProjection;float4x4 matView;float4x4 matWorldView;float4 fLightPos;struct VS_INPUT {   float4 Position : POSITION0;   float4 Normal : NORMAL0;   float4 TexCoord : TEXCOORD0;   };struct VS_OUTPUT {   float4 Position : POSITION0;   float4 TexCoord : TEXCOORD0;   };VS_OUTPUT vs_main( VS_INPUT Input ){   VS_OUTPUT Output;   Output.Position = mul( Input.Position, matViewProjection);   Output.TexCoord = Input.TexCoord;      fLightPos = float4(0,2,10,1);   float4 lightDir = Input.Position - fLightPos;   lightDir.w = 0;   Input.Normal.w = 0;   lightDir = mul(lightDir, matView);   Input.Normal = mul(Input.Normal, matView);      float x = max(dot(lightDir,Input.Normal),0);      //Output.TexCoord = Input.TexCoord;   Output.TexCoord.x = x;   Output.TexCoord.y = 0.5f;      return( Output );   }
I meant matWorld or matWorldView, I got confused as to what mat was [grin] - don't just use matView.

What space is your light position in? World space? Object space? View space?
agi, it is in World space. Also, when I switch to my WorldView matrix, I still get the same prob. Any other ideas?
- don't multiply lightDir by any matrix
- only multiply Input.Normal by matworld (to get it into world space, same as light)
- before you subtract to find lightDir, multiply Input.position by matWorld (to get the vertex position into world space, same as light)
Agi, I followed your suggestions and now instead of my mesh being a dark grey , near 0, it looks like my texture isn't being used at all anymore. That or my mesh is now transparent haha. Any other ideas? Thanks for your help btw!
float4x4 matViewProjection;float4x4 matView;float4x4 matWorld;float4x4 matWorldView;float4 fLightPos;struct VS_INPUT {   float4 Position : POSITION0;   float4 Normal : NORMAL0;   float4 TexCoord : TEXCOORD0;   };struct VS_OUTPUT {   float4 Position : POSITION0;   float4 TexCoord : TEXCOORD0;   };VS_OUTPUT vs_main( VS_INPUT Input ){   VS_OUTPUT Output;   Output.Position = mul( Input.Position, matWorld);   Output.TexCoord = Input.TexCoord;      fLightPos = float4(0,2,10,1);   float4 lightDir = Input.Position - fLightPos;   lightDir.w = 0;   Input.Normal.w = 0;   //lightDir = mul(lightDir, matWorldView);   Input.Normal = mul(Input.Normal, matWorld);      float x = max(dot(lightDir,Input.Normal),0);      //Output.TexCoord = Input.TexCoord;   Output.TexCoord.x = x;   Output.TexCoord.y = 0.5f;      return( Output );   }


I meant to use matWorld when you find the light direction, not the final clip-space position:

// multiply by matWorldViewProjection when you use it hereOutput.Position = mul( Input.Position, matWorld);   Output.TexCoord = Input.TexCoord;   

   fLightPos = float4(0,2,10,1);// multiply by matWorld when you use it here   float4 lightDir = Input.Position - fLightPos;
Agi, this gives me the same effect.

float4x4 matViewProjection;float4x4 matView;float4x4 matWorld;float4x4 matWorldView;float4 fLightPos;struct VS_INPUT {   float4 Position : POSITION0;   float4 Normal : NORMAL0;   float4 TexCoord : TEXCOORD0;   };struct VS_OUTPUT {   float4 Position : POSITION0;   float4 TexCoord : TEXCOORD0;   };VS_OUTPUT vs_main( VS_INPUT Input ){   VS_OUTPUT Output;   Output.Position = mul( Input.Position, matWorldViewProjection);   Output.TexCoord = Input.TexCoord;   fLightPos = float4(1,2,10,1);   float4 Pos = mul( Input.Position, matWorld);   float4 lightDir = Pos - fLightPos;   lightDir.w = 0;   Input.Normal.w = 0;      //lightDir = mul(lightDir, matWorldView);   Input.Normal = mul(Input.Normal, matWorld);      float x = max(dot(lightDir,Input.Normal),0);      //Output.TexCoord = Input.TexCoord;   Output.TexCoord.x = x;   Output.TexCoord.y = 0.5f;      return( Output );   }

This topic is closed to new replies.

Advertisement