Jump to content
  • Advertisement
Sign in to follow this  

Normal vectors hate me

This topic is 3987 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I'm writing a simple graphics program using DirectX 10 (though the Direct3D version shouldn't matter as far as my problem is concerned). My program can successfully display 3D models on screen using vertex and pixel shaders (written in HLSL). My problem is that no matter what I do, I can't seem to transform my models' normal vectors to make the lighting calculations work properly. My test model is a simple textured cube and I know that the initial vertex normals are correct. I also know that my transformation matrices are correct as I can successfully translate and rotate the cube. When it comes to transforming the normals, however, nothing seems to work. I've tried using the same world transformation matrix as the vertices; it didn't work. I've tried using the inverse transpose world matrix (as well as the transpose inverse world matrix); it didn't work. I know it must be the transformed normals because when I try lighting the cube without moving it around first, the lighting model appears to shade the cube in an expected way. Here is the relevent code: Calculating the transpose inverse world matrix (I've tried it the other way too)
	D3DXMATRIX invmat;
	D3DXMATRIX transpose;
	D3DXMatrixTranspose(&transpose, &transformation.trans);
	//D3DXMatrixTranspose(&transpose, &invmat);
Shader code:
//constant buffer of external variables
cbuffer variables
	matrix Projection;
	matrix View;
	matrix World;
	matrix InvWorld;
	float Specular_exponent;
	float3 Specular;
	float3 Emissive;
	float3 local_light_pos;
	float local_light_radius;
	float3 local_light_color;

extern Texture2D EmissiveTexture;
extern Texture2D BumpmapTexture;
extern Texture2D DiffuseTexture;
extern Texture2D SpecularTexture;
extern Texture2D TransparencyTexture;

float3x3 TBNMatrix;

float3 color = float3(1.0f, 0.0f, 0.0f);

//PS_INPUT - input variables to the pixel shader
struct PS_INPUT
	float4 Pos : SV_POSITION;
	float2 Tex : TEXCOORD0;
	float3 Normal : TEXCOORD1;
	float4 Orig_Pos : TEXCOORD2;
	float3 LightDir : TEXCOORD3;

//texture sampler
SamplerState samLinear
	AddressU = Wrap;
	AddressV = Wrap;

//attenuation function (no bump-mapping)
float Attenuation(float3 LightPos, float3 Pos3D, float light_radius)
	float distance = distance(LightPos, Pos3D);
	distance = distance * (1 / light_radius);

	float atten = 1 / (distance + (distance * distance));
	return atten;

//test vertex shader
PS_INPUT test_vs(float4 Pos : POSITION, float2 Tex : TEXCOORD0, float3 Normal : NORMAL, float3 tangent : TANGENT, float3 bitangent : BITANGENT)
	PS_INPUT psInput;

	float4x4 WorldView = mul(World, View);
	float4x4 WorldViewProj = mul(WorldView, Projection);

	psInput.Pos = mul(Pos, WorldViewProj);
	psInput.Orig_Pos = mul(Pos, World);
	psInput.Tex = Tex;

	psInput.Normal = mul(InvWorld, float4(Normal,0.0f)).xyz;
	psInput.Normal = normalize(psInput.Normal);

	psInput.LightDir = normalize(local_light_pos - psInput.Orig_Pos);

	TBNMatrix = float3x3(tangent,bitangent,psInput.Normal);

	return psInput;

//test pixel shader
float4 test_ps(PS_INPUT psInput) : SV_TARGET

	float DiffuseLightingFactor = dot(psInput.LightDir, psInput.Normal);
	float atten = Attenuation(local_light_pos, psInput.Orig_Pos, local_light_radius);

	float4 outColor;
	//outColor.rgb = DiffuseTexture.Sample(samLinear, psInput.Tex) * local_light_color * (1-DiffuseLightingFactor) * atten;
	outColor.rgb = local_light_color * (1-DiffuseLightingFactor) * atten;

	outColor.a = 1.0f;

	return outColor;

//define a technique
technique10 Render
	pass P0
		SetVertexShader(CompileShader(vs_4_0, test_vs()));
		SetPixelShader(CompileShader(ps_4_0, test_ps()));
EDIT: Please remember to use 'source' tags in future. [Edited by - jollyjeffers on January 14, 2008 5:21:56 AM]

Share this post

Link to post
Share on other sites
I'm not on my D3D10 devbox right now, but I'm pretty sure the code I was playing with over the weekend used the mul(Normal,(float3x3)World) approach - Provided you ignore the translation elements (the cast to 3x3 does) and re-normalize (to avoid any scaling) you should be fine.

Give that a try - you might find some examples if you search around online.


Share this post

Link to post
Share on other sites
Sorry about the lack of source tags (and the silly name of my post). I've been so frustrated with this problem that I became a member just so I could make this post and I'm afraid I neglected some of the forum rules. I'm frustrated precisely because everything I've read on the internet seems to suggest that my code is correct and yet it still doesn't seem to be working (even your suggestion didn't fix the problem, unfortunately). Maybe I can better explain what my program is doing: I have a cube at the origin and a per-pixel light positioned above and to the right (the exact coordinates are 4.5,4.5,0.0). I've programmed per-pixel shader lights before, but always with static scenes. In this scene the cube rotates about the y-axis and the light never moves, very simple. Yet no matter how I transform the cube's normals the cube isn't lit the right way. It should be lit in such a way that the part of the cube closest to the light (whichever part that might currently be) should be brighter while the parts farther from the light should be darker. Instead, the lightness and darkness of the sides seems to circulate regardless of the sides' actual proximity to the light. The most obvious example of this is the top of the cube, which should always be well lit given how close it is to the light. Instead it becomes lighter, darker, lighter, darker, etc... I'm at my wit's end. This is literaly the only thing holding me up in this program (everything else seems to work fine). Any help would be greatly appreciated. I'd like to show my actual program as it runs but I don't know how to make the executable available (I doubt the showcase was intended for this :) ). Thank you for your time and consideration.

Share this post

Link to post
Share on other sites
How many vertices do you use for the cube? If it's 8, the normals are shared between the faces (so to speak), and will produce a smoothed out look. If it's 24, it should look okay.

Also, why are you sure that the normals are correct? Can you post the code that initializes them?

Share this post

Link to post
Share on other sites
jollyjeffers seems to be right. You are trying to do lighting in a world space right? (despite some code fragments suggesting tangent space...). So why world inverse transform? You want to use World transform without translation and scale, thus upper 3x3 and then normalize.

For simplicity start with identity world matrix. If your cube looks right, try rotation/scale. If your cube looks wrong with identity WM, problem lies elsewhere.

Check also small problem with diffuselightingfactor - should be clamped.


Share this post

Link to post
Share on other sites
I think I've discovered the problem. First, some background info: I created the cube model using the Anim8tor modeling program. I read the model data into my own custom file format using a program I wrote. The file writing program requires that the Anim8tor model be exported to a C file (Anim8tor has this capability). Well, to make a long story short, when Anim8tor exported my model to a C file, the vertices which should have had normals along the y-axis were given normals along the x-axis, and vice-versa. When I swapped them (and negated them to deal with handed-ness issues), the lighting calculations worked just as they should. So it seems the problem all along was with my modeling program; not exactly a good situation but progress is progress. Thanks to everyone for your input. I suppose the logical question now would be to ask for advice about Anim8tor, so if anyone has encountered this problem before please let me know how you solved it. Thanks.

Share this post

Link to post
Share on other sites
Sign in to follow this  

  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!