# Normal vectors hate me

This topic is 3987 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

I'm writing a simple graphics program using DirectX 10 (though the Direct3D version shouldn't matter as far as my problem is concerned). My program can successfully display 3D models on screen using vertex and pixel shaders (written in HLSL). My problem is that no matter what I do, I can't seem to transform my models' normal vectors to make the lighting calculations work properly. My test model is a simple textured cube and I know that the initial vertex normals are correct. I also know that my transformation matrices are correct as I can successfully translate and rotate the cube. When it comes to transforming the normals, however, nothing seems to work. I've tried using the same world transformation matrix as the vertices; it didn't work. I've tried using the inverse transpose world matrix (as well as the transpose inverse world matrix); it didn't work. I know it must be the transformed normals because when I try lighting the cube without moving it around first, the lighting model appears to shade the cube in an expected way. Here is the relevent code: Calculating the transpose inverse world matrix (I've tried it the other way too)
	D3DXMATRIX invmat;
D3DXMATRIX transpose;
D3DXMatrixTranspose(&transpose, &transformation.trans);
D3DXMatrixInverse(&invmat,NULL,&transpose);
//D3DXMatrixInverse(&invmat,NULL,&transformation.trans);
//D3DXMatrixTranspose(&transpose, &invmat);
r_elem-&gt;invworld-&gt;SetMatrix((float*)&invmat);

//constant buffer of external variables
cbuffer variables
{
matrix Projection;
matrix View;
matrix World;
matrix InvWorld;
float Specular_exponent;
float3 Specular;
float3 Emissive;
float3 local_light_pos;
float3 local_light_color;
};

extern Texture2D EmissiveTexture;
extern Texture2D BumpmapTexture;
extern Texture2D DiffuseTexture;
extern Texture2D SpecularTexture;
extern Texture2D TransparencyTexture;

float3x3 TBNMatrix;

float3 color = float3(1.0f, 0.0f, 0.0f);

//PS_INPUT - input variables to the pixel shader
struct PS_INPUT
{
float4 Pos : SV_POSITION;
float2 Tex : TEXCOORD0;
float3 Normal : TEXCOORD1;
float4 Orig_Pos : TEXCOORD2;
float3 LightDir : TEXCOORD3;
};

//texture sampler
SamplerState samLinear
{
Filter = MIN_MAG_MIP_LINEAR;
};

//attenuation function (no bump-mapping)
float Attenuation(float3 LightPos, float3 Pos3D, float light_radius)
{
float distance = distance(LightPos, Pos3D);
distance = distance * (1 / light_radius);

float atten = 1 / (distance + (distance * distance));
return atten;
}

PS_INPUT test_vs(float4 Pos : POSITION, float2 Tex : TEXCOORD0, float3 Normal : NORMAL, float3 tangent : TANGENT, float3 bitangent : BITANGENT)
{
PS_INPUT psInput;

float4x4 WorldView = mul(World, View);
float4x4 WorldViewProj = mul(WorldView, Projection);

psInput.Pos = mul(Pos, WorldViewProj);
psInput.Orig_Pos = mul(Pos, World);
psInput.Tex = Tex;

psInput.Normal = mul(InvWorld, float4(Normal,0.0f)).xyz;
psInput.Normal = normalize(psInput.Normal);

psInput.LightDir = normalize(local_light_pos - psInput.Orig_Pos);

TBNMatrix = float3x3(tangent,bitangent,psInput.Normal);

return psInput;
}

float4 test_ps(PS_INPUT psInput) : SV_TARGET
{

float DiffuseLightingFactor = dot(psInput.LightDir, psInput.Normal);
float atten = Attenuation(local_light_pos, psInput.Orig_Pos, local_light_radius);

float4 outColor;
//outColor.rgb = DiffuseTexture.Sample(samLinear, psInput.Tex) * local_light_color * (1-DiffuseLightingFactor) * atten;
outColor.rgb = local_light_color * (1-DiffuseLightingFactor) * atten;

outColor.a = 1.0f;

return outColor;
}

//define a technique
technique10 Render
{
pass P0
{
}
}

EDIT: Please remember to use 'source' tags in future. [Edited by - jollyjeffers on January 14, 2008 5:21:56 AM]

##### Share on other sites
I'm not on my D3D10 devbox right now, but I'm pretty sure the code I was playing with over the weekend used the mul(Normal,(float3x3)World) approach - Provided you ignore the translation elements (the cast to 3x3 does) and re-normalize (to avoid any scaling) you should be fine.

Give that a try - you might find some examples if you search around online.

hth
Jack

##### Share on other sites
Sorry about the lack of source tags (and the silly name of my post). I've been so frustrated with this problem that I became a member just so I could make this post and I'm afraid I neglected some of the forum rules. I'm frustrated precisely because everything I've read on the internet seems to suggest that my code is correct and yet it still doesn't seem to be working (even your suggestion didn't fix the problem, unfortunately). Maybe I can better explain what my program is doing: I have a cube at the origin and a per-pixel light positioned above and to the right (the exact coordinates are 4.5,4.5,0.0). I've programmed per-pixel shader lights before, but always with static scenes. In this scene the cube rotates about the y-axis and the light never moves, very simple. Yet no matter how I transform the cube's normals the cube isn't lit the right way. It should be lit in such a way that the part of the cube closest to the light (whichever part that might currently be) should be brighter while the parts farther from the light should be darker. Instead, the lightness and darkness of the sides seems to circulate regardless of the sides' actual proximity to the light. The most obvious example of this is the top of the cube, which should always be well lit given how close it is to the light. Instead it becomes lighter, darker, lighter, darker, etc... I'm at my wit's end. This is literaly the only thing holding me up in this program (everything else seems to work fine). Any help would be greatly appreciated. I'd like to show my actual program as it runs but I don't know how to make the executable available (I doubt the showcase was intended for this :) ). Thank you for your time and consideration.

##### Share on other sites
How many vertices do you use for the cube? If it's 8, the normals are shared between the faces (so to speak), and will produce a smoothed out look. If it's 24, it should look okay.

Also, why are you sure that the normals are correct? Can you post the code that initializes them?

##### Share on other sites
jollyjeffers seems to be right. You are trying to do lighting in a world space right? (despite some code fragments suggesting tangent space...). So why world inverse transform? You want to use World transform without translation and scale, thus upper 3x3 and then normalize.

For simplicity start with identity world matrix. If your cube looks right, try rotation/scale. If your cube looks wrong with identity WM, problem lies elsewhere.

Check also small problem with diffuselightingfactor - should be clamped.

M.

##### Share on other sites
I think I've discovered the problem. First, some background info: I created the cube model using the Anim8tor modeling program. I read the model data into my own custom file format using a program I wrote. The file writing program requires that the Anim8tor model be exported to a C file (Anim8tor has this capability). Well, to make a long story short, when Anim8tor exported my model to a C file, the vertices which should have had normals along the y-axis were given normals along the x-axis, and vice-versa. When I swapped them (and negated them to deal with handed-ness issues), the lighting calculations worked just as they should. So it seems the problem all along was with my modeling program; not exactly a good situation but progress is progress. Thanks to everyone for your input. I suppose the logical question now would be to ask for advice about Anim8tor, so if anyone has encountered this problem before please let me know how you solved it. Thanks.

1. 1
2. 2
Rutin
16
3. 3
4. 4
5. 5

• 26
• 9
• 11
• 9
• 9
• ### Forum Statistics

• Total Topics
633710
• Total Posts
3013486
×