Sign in to follow this  
d h k

[D3D9/HLSL] Simple diffuse light shader...

Recommended Posts

I'm trying to achieve a simple diffuse light effect through HLSL vertex and pixel shader. My shader code is pretty much taken from a Gamasutra article, just slightly modified, take a look at this .fx file:
float4x4 matWorldViewProj;
float4x4 matWorld;
float4 vecLightDir;

struct VS_OUTPUT
{
    float4 Pos : POSITION;
    float2 UV : TEXCOORD0;
    float3 Light : TEXCOORD1;
    float3 Norm : TEXCOORD2;
};

VS_OUTPUT VS(float4 Pos : POSITION, float2 UV : TEXCOORD0, float3 Normal : NORMAL)
{
    VS_OUTPUT Out = (VS_OUTPUT)0;
    Out.Pos = mul(Pos, matWorldViewProj); // transform Position
    Out.UV = UV; // pass UV coordinates on
    Out.Light = normalize(vecLightDir); // output light vector
    Out.Norm = normalize(mul(Normal, matWorld)); // transform Normal and normalize it
    return Out;
}

float4 PS(float2 UV : TEXCOORD0, float3 Light : TEXCOORD1, float3 Norm : TEXCOORD2, sampler2D tex0) : COLOR
{
    float4 diffuse = tex2D ( tex0, UV );
    float4 ambient = {0.05, 0.05, 0.075, 1.0};
    return ambient + diffuse * saturate(dot(Light, Norm));
}

technique EntryPoint
{
    pass SinglePass
    {
        VertexShader = compile vs_2_0 VS ( );
	PixelShader = compile ps_2_0 PS ( );
    }
}


As you can see, I pretty much only added texture mapping (btw. this didn't make a difference, the results with the original code from the article and my modified version yield the same lighting results/problems). My drawing code in the application looks like this:
void Render ( )
{
	unsigned int num_passes;

	// clear the back- and z-buffer
	device->Clear ( 0, NULL, D3DCLEAR_TARGET | D3DCLEAR_ZBUFFER, D3DCOLOR_XRGB ( clear_color.r, clear_color.g, clear_color.b ), 1.0f, 0 );

	effect->Begin ( &num_passes, 0 );

	for ( unsigned int pass = 0; pass < num_passes; pass++ )
	// loop through all passes
	{
		effect->BeginPass ( pass );

		DWORD old_fvf = NULL;
	
		// get current flexible vertex format
		device->GetFVF ( &old_fvf );

		// set format for our models
		device->SetFVF ( D3DFVF_XYZ | D3DFVF_NORMAL | D3DFVF_TEX2 );

		std::vector<Model*>::iterator i;
		for ( i = model.begin ( ); i < model.end ( ); i++ )
		// loop through all models
		{
			// apply the matrix transformations for the model
			(*i)->ApplyMatrixTransformations ( device );
				
			// now get combined matrix (ie. the world-view-projection matrix)
			D3DXMATRIX combined_matrix = GetCombinedMatrix ( );

			effect->SetMatrix ( "matWorldViewProj", &combined_matrix );

			D3DXMATRIX world_matrix;

			// get world matrix
			device->GetTransform ( D3DTS_WORLD, &world_matrix );

			effect->SetMatrix ( "matWorld", &world_matrix );

			D3DXVECTOR4 light_vector ( 0.0f, 0.0f, 1.0f, 0.0f );

			effect->SetVector ( "vecLightDir", &light_vector );

			// make sure all values send to the shader are updated
			effect->CommitChanges ( );

			// render model
			(*i)->Render ( device );
		}

		// reset to old format
		device->SetFVF ( old_fvf );

		effect->EndPass ( );
	}

	effect->End ( );
}


As you can see, at the moment, I just hardcode the light vector so I should get a simple direction light. What I get, however, is relatively weird:
These are two shots of my 3-test-cubes model. If nothing is wrong with the shader (I'd need some confirmation there), then it has to be the normals, I guess. I calculate them using the lib3ds functionality, like this:
for ( int i = 0; i < file->nmeshes; i++ )
// loop through all meshes+
{
	float ( *normals )[3] = ( float (*)[3] ) malloc ( 3 * 3 * sizeof ( float ) * file->meshes[i]->nfaces );

	// calculate the normals for all vertices in the mesh
	lib3ds_mesh_calculate_vertex_normals ( file->meshes[i], normals );
}
Then, when I create my vertex buffer, I retrieve the proper normals from the "normals" array and add them to the vertex buffer. I also debug display the normals from that array when I'm loading the model and they look very correct, check out this output for one of the cubes (ie. one submesh):
Mesh 0 includes 26 vertices...

Format:
i:  x, y, z               / u, v / nx, ny, nz

0: -9.999, -9.999, 0.001 / 1, 1 / 0, 0, -1
1: 10.001, -9.999, 0.001 / 0, 1 / 0, 0, -1
2: -9.999, 10.001, 0.001 / 1, 0 / 0, 0, -1
3: 10.001, 10.001, 0.001 / 0, 0 / 0, 0, -1
4: -9.999, -9.999, 20.001 / 0, 1 / 0, 0, -1
5: 10.001, -9.999, 20.001 / 1, 1 / 0, 0, -1
6: -9.999, 10.001, 20.001 / 0, 0 / 0, 0, 1
7: 10.001, 10.001, 20.001 / 1, 0 / 0, 0, 1
8: -9.999, -9.999, 0.001 / 0, 1 / 0, 0, 1
9: 10.001, -9.999, 0.001 / 1, 1 / 0, 0, 1
10: 10.001, -9.999, 20.001 / 1, 0 / 0, 0, 1
11: 10.001, -9.999, 20.001 / 1, 0 / 0, 0, 1
12: -9.999, -9.999, 20.001 / 0, 0 / 0, -1, 0
13: -9.999, -9.999, 0.001 / 0, 1 / 0, -1, 0
14: 10.001, 10.001, 0.001 / 1, 1 / 0, -1, 0
15: 10.001, -9.999, 20.001 / 0, 0 / 0, -1, 0
16: 10.001, 10.001, 0.001 / 0, 1 / 0, -1, 0
17: -9.999, 10.001, 0.001 / 1, 1 / 0, -1, 0
18: -9.999, 10.001, 20.001 / 1, 0 / 1, 0, 0
19: -9.999, 10.001, 20.001 / 1, 0 / 1, 0, 0
20: 10.001, 10.001, 20.001 / 0, 0 / 1, 0, 0
21: 10.001, 10.001, 0.001 / 0, 1 / 1, 0, 0
22: -9.999, 10.001, 0.001 / 0, 1 / 1, 0, 0
23: -9.999, -9.999, 20.001 / 1, 0 / 1, 0, 0
24: -9.999, -9.999, 20.001 / 1, 0 / 0, 1, 0
25: -9.999, 10.001, 0.001 / 0, 1 / 0, 1, 0
As I said, looks fine to me. So what could be wrong? Any help is appreciated, as always and thanks for your time.

Share this post


Link to post
Share on other sites
Shouldn't you be using D3DFVF_TEX1 instead of D3DFVF_TEX2?

EDIT: I've also just noticed you are normalising your light vector and normals in the vertex shader. This should really be done in the pixel shader. Although I don't think it's going to fix your output.

[Edited by - adt7 on August 19, 2009 9:12:14 AM]

Share this post


Link to post
Share on other sites
Hm. Maybe you are accidentally translating your normal. Does this make a difference?

Out.Norm = normalize(mul(Normal, (float3x3)matWorld));

Share this post


Link to post
Share on other sites
When you are done making your mesh (vertex buffer), call ComputeNormals on it. Make sure the vertex declaration of the vertex buffer is accurate.

Some off the wall questions, does your video card handle Shader 2.0? It doesn't look like you're doing anything that requires shader 2, so compile to shader 1 just to see. If still there, go back to 2. Just a thought.

Are you on a nVidia, ATI or Intel card? I know that when declaring semantics and such, things get funky between the 3 cards... nVidia is generally more intelligent when handling shaders (IMO, but that's a different thread altogether).

Also, try the shader in FX Composer. Does it look correct? If it does, the issue is in the app code. If it does not, the issue is in the shader.

Just trying to give you some new angles of approach.

Share this post


Link to post
Share on other sites
One other thing...

I've never passed a sampler as a parameter before. Try declaring it within the shader something like:

texture diffuseTexture : Diffuse
<
string ResourceName = "default_color.dds";
>;

sampler TextureSampler = sampler_state
{
texture = <diffuseTexture>;
AddressU = CLAMP;
AddressV = CLAMP;
AddressW = CLAMP;
MIPFILTER = LINEAR;
MINFILTER = LINEAR;
MAGFILTER = LINEAR;
};

then remove the sampler param and replace
float4 diffuse = tex2D ( tex0, UV );

with
float4 diffuse = tex2D ( TextureSampler, UV );

Other than that, the shader runs fine.

Share this post


Link to post
Share on other sites
Quote:
Original post by EnochDagor
When you are done making your mesh (vertex buffer), call ComputeNormals on it.


How exactly would I do that? Call ComputeNormals on what? I couldn't find much about this, the MSDN only mentions this as a function for .x meshes, I don't use these?

Quote:
Original post by EnochDagor
Make sure the vertex declaration of the vertex buffer is accurate.


Well, as you can see, before rendering the model I call:


device->SetFVF ( D3DFVF_XYZ | D3DFVF_NORMAL | D3DFVF_TEX1 );


And when I create the vertex buffer for a model, I call that like this:


// create the vertex buffer
device->CreateVertexBuffer ( sizeof ( Vertex ) * total_vertices, 0, D3DFVF_XYZ | D3DFVF_NORMAL | D3DFVF_TEX1, D3DPOOL_MANAGED, &vertex_buffer, NULL );


Same FVF, is that what you mean by accurate?


Quote:
Original post by EnochDagor
Some off the wall questions, does your video card handle Shader 2.0? It doesn't look like you're doing anything that requires shader 2, so compile to shader 1 just to see. If still there, go back to 2. Just a thought.

Are you on a nVidia, ATI or Intel card? I know that when declaring semantics and such, things get funky between the 3 cards... nVidia is generally more intelligent when handling shaders (IMO, but that's a different thread altogether).


Running an NVidia GeForce GTX 260, so shader model 2 isn't a problem.


About the rest: I can't really have the texture name in the shader for my purposes, also the texture mapping does work absolutely fine, so I'm not convinced that this is the cause for my problem, to be honest. I don't want to sound disrespectful and I'm thankful for your time and effort and the good points you make, but I just don't see how that could be related to my normal/lighting problem.

Share this post


Link to post
Share on other sites
Dang, that was from an old clipboard or something, it already is TEX1, I don't use TEX2 anywhere in the project. Sorry for the confusion.

Share this post


Link to post
Share on other sites
This:


D3DXMATRIX GetCombinedMatrix ( void )
{
D3DXMATRIX world_matrix, view_matrix, projection_matrix, combined_matrix;

// get world-, view- and projection matrix
device->GetTransform ( D3DTS_WORLD, &world_matrix );
device->GetTransform ( D3DTS_VIEW, &view_matrix );
device->GetTransform ( D3DTS_PROJECTION, &projection_matrix );

// multiply them together into a combined matrixx
D3DXMatrixMultiply ( &combined_matrix, &world_matrix, &view_matrix );
D3DXMatrixMultiply ( &combined_matrix, &combined_matrix, &projection_matrix );

return combined_matrix;
}

Share this post


Link to post
Share on other sites
Ok, that looks good. I placed your shader in FX Composer and removed the parameterized sampler (cuz FX composer barked at it). It works fine and lights as expected.

Can you paste the code for your box creation?

Share this post


Link to post
Share on other sites
Quote:
Original post by EnochDagor
Ok, that looks good. I placed your shader in FX Composer and removed the parameterized sampler (cuz FX composer barked at it). It works fine and lights as expected.


Thanks, good to know this isn't the problem. So then I guess it IS the normals or something along these lines.

Quote:

Can you paste the code for your box creation?


Well, I modeled them in 3d Studio Max and export it as a .3ds into the application using the lib3ds. Here's the code for that:


void Model::CreateVertexBuffer ( IDirect3DDevice9 *device )
{
void *void_pointer;

// create a temporary buffer for the vertices
Vertex *vertices = new Vertex[total_vertices];

int counter = 0;
for ( int i = 0; i < file->nmeshes; i++ )
// loop through all meshes+
{
float ( *normals )[3] = ( float (*)[3] ) malloc ( 3 * 3 * sizeof ( float ) * file->meshes[i]->nfaces );

// calculate the normals for all vertices in the mesh
lib3ds_mesh_calculate_vertex_normals ( file->meshes[i], normals );

std::stringstream ss;
ss << "Mesh " << i << " includes " << file->meshes[i]->nvertices << " vertices..." << std::endl << std::endl;

for ( int j = 0; j < file->meshes[i]->nvertices; j++ )
// loop through all vertices of this mesh
{
Vertex temp ( file->meshes[i]->vertices[j][0], file->meshes[i]->vertices[j][1], file->meshes[i]->vertices[j][2] );

// also parse texture coordinates from file
temp.u = file->meshes[i]->texcos[j][0];
temp.v = file->meshes[i]->texcos[j][1];

// we actually need to invert the v-coordinate
temp.v = 1.0f - temp.v;

// get the normal for this vertex
temp.normal.x = normals[j][0];
temp.normal.y = normals[j][1];
temp.normal.z = normals[j][2];

if ( j < 50 )
// for the first few vertices
ss << counter << ": " << temp.x << ", " << temp.y << ", " << temp.z << " / " << temp.u << ", " << temp.v
<< " / " << temp.normal.x << ", " << temp.normal.y << ", " << temp.normal.z << std::endl;

// copy data from the model to the temporary buffer
vertices[counter] = temp;

counter++;
}

MessageBox ( 0, ss.str ( ).c_str ( ), NULL, NULL );
}

// create the vertex buffer
device->CreateVertexBuffer ( sizeof ( Vertex ) * total_vertices, 0, D3DFVF_XYZ | D3DFVF_NORMAL | D3DFVF_TEX1, D3DPOOL_MANAGED, &vertex_buffer, NULL );

// copy the contents of the temporary buffer into the vertex buffer
vertex_buffer->Lock ( 0, 0, ( void **) &void_pointer, 0 );
memcpy ( void_pointer, vertices, sizeof ( Vertex ) * total_vertices );
vertex_buffer->Unlock ( );

delete [] vertices;
}

void Model::CreateIndexBuffer ( IDirect3DDevice9 *device )
{
void *void_pointer;

// create a temporary buffer for the indices
Index *indices = new Index[total_faces];

int counter = 0;
for ( int i = 0; i < file->nmeshes; i++ )
// loop through all meshes
{
int offset = 0;

std::stringstream ss;
ss << "Mesh " << i << " includes " << file->meshes[i]->nfaces << " faces..." << std::endl << std::endl;

if ( i > 0 )
// if we are not dealing with the first mesh
{
for ( int k = 0; k < i; k++ )
// loop through all the meshes that come before this one
// add up their number of vertices
offset += file->meshes[k]->nvertices;
}

for ( int j = 0; j < file->meshes[i]->nfaces; j++ )
// loop through all faces of this mesh
{
Index temp ( file->meshes[i]->faces[j].index[0] + offset, file->meshes[i]->faces[j].index[1] + offset, file->meshes[i]->faces[j].index[2] + offset );

ss << counter << ": " << temp.a << ", " << temp.b << ", " << temp.c << std::endl;

// copy data from the model to the temporary buffer
indices[counter] = temp;

counter++;
}

//MessageBox ( 0, ss.str ( ).c_str ( ), NULL, NULL );
}

// create the index buffer
device->CreateIndexBuffer ( sizeof ( Index ) * total_faces, 0, D3DFMT_INDEX16, D3DPOOL_MANAGED, &index_buffer, NULL );

// copy the contents of the temporary buffer into the index buffer
index_buffer->Lock ( 0, 0, ( void **) &void_pointer, 0 );
memcpy ( void_pointer, indices, sizeof ( Index ) * total_faces );
index_buffer->Unlock ( );

delete [] indices;
}

bool Model::Load ( std::string filename, IDirect3DDevice9 *device )
{
file = lib3ds_file_open ( filename.c_str ( ) );

if ( !file )
return false;

// reset vertex and face counter
total_vertices = 0;
total_faces = 0;

for ( int i = 0; i < file->nmeshes; i++ )
// loop through all meshes
{
// add the vertices of this mesh to our count
total_vertices += file->meshes[i]->nvertices;

// add the faces of this mesh to our count
total_faces += file->meshes[i]->nfaces;
}

std::stringstream ss;
ss << "Loading model '" << filename << "'..." << std::endl;
ss << "Number of meshes: " << file->nmeshes << "..." << std::endl;
ss << "Number of total vertices: " << total_vertices << "..." << std::endl;
ss << "Number of total faces: " << total_faces << "...";
//MessageBox ( 0, ss.str ( ).c_str ( ), NULL, NULL );

// create the vertex- and index buffer
CreateVertexBuffer ( device );
CreateIndexBuffer ( device );

return true;
}



Share this post


Link to post
Share on other sites
I noticed you use the 3ds library. Does that assume a right-handed vs. left-handed coordinate system? I believe studio max is right-handed. DirectX is left-handed. That would effect the direction of the normals and cause your shading to look funky (which it does).

Share this post


Link to post
Share on other sites
Quote:
Original post by Buckeye
I noticed you use the 3ds library. Does that assume a right-handed vs. left-handed coordinate system? I believe studio max is right-handed. DirectX is left-handed. That would effect the direction of the normals and cause your shading to look funky (which it does).


Interesting. I tried to invert the .x part of the normal that I read from the model when I assign that to the vertex (ie. temp.normal.x = normals[j][0] * -1;) but it didn't help. Looks the same in fact. Is there something else to do when converting from right- to left-handed?

Quote:
Original post by EnochDagor
Ok, the next step would be to load a mesh that is known to work (if you have one). If not, I can give you some code to create one and try it.


Sure, go ahead!

Thanks so far, guys.

Share this post


Link to post
Share on other sites
Quote:
I tried to invert the .x part of the normal

Converting from right to left is a little complicated. With regard to the normals, after you load the mesh, you can use D3DXComputeNormals and let it figure things out.

I can't tell from the images, but another problem that occurs with right vs left is that texture coords need to be converted - something like tu=tu but tv = 1.0-tv (I can't remember for sure). And I can't remember whether that's resolved or not if you unwind the vertices (see below). Better try other things first and resolve tex coords later.

Also, you may need to "unwind" the vertices (which, from your data above, appear to be CCW rather than CW) either by changing the order of the vertices for each face, or changing the indices for each triangle to render then CW (which DirectX expects). If you unwind the vertices, do that before you compute normals.

Share this post


Link to post
Share on other sites
But how do I call D3DXComputeNormals when I don't have an LPD3DXBASEMESH? Do I need to create one somehow just for the function?

I actually noticed the tex coord problem and inverted the v coordinate just as you said.

Share this post


Link to post
Share on other sites
Looks like a problem I had when I was using X files and I had incorrect adjacency data before asking D3DX to compute normals for me. You should either dig into how lib_3ds_ gets its adjacency data to compute those normals for you or compute them yourself.

Share this post


Link to post
Share on other sites
Looks like a problem I had when I was using X files and I had incorrect adjacency data before asking D3DX to compute normals for me. You should either dig into how lib_3ds_ gets its adjacency data to compute those normals for you or compute them yourself.

EDIT:

I had a look at your FX file again and saw some possible mistakes:

1. Try changing the code that transforms your normal to world space in the vertex shader as follows:

Out.Norm = normalize(mul(float4(Normal, 0.0f), matWorld));

2. Renormalize the normal upon entry into the pixel shader:

Norm = normalize(Norm);

3. Compute color as follows:

return ambient + (diffuse * max(dot(-Light, Norm), 0.0f));

Share this post


Link to post
Share on other sites
Quote:
how do I call D3DXComputeNormals when I don't have an LPD3DXBASEMESH?
You can't[ATTENTION][SMILE] Bad assumption on my part.

You'd have to duplicate the function based on how you store the model information. If you want smooth normals (normals which will "smooth" the rendering between adjacent faces), that's a huge pain. You need (or would have to generate) adjacency information for the model.

Computing "flat" normals for each vertex:

for each triangle: // 3 vertices in CW order, vertex0 through vertex2
vector1 = vector( vertex1-vertex0 );
vector2 = vector( vertex2-vertex0 );
normal = crossproduct(vector2, vector1);
normalize(normal);
vertex0.normal = vertex1.normal = vertex2.normal = normal;

If that doesn't work, try normal = crossproduct(vector1, vector2). I'm getting old and can't remember everything.[WINK]

Share this post


Link to post
Share on other sites
Because I am not good at C++, here's the pseudo code to make a mesh... this is a brute force method. There may be another way to do this... but this guarantees the model's data is accurate:

1) Instantiate a mesh = Mesh.Box(). We'll call it oMesh.
2) Call ComputeNormals() on the oMesh object
3) DX defaults to Position and Normal, we need to add textured
4) Instantiate a second mesh with the params NumberFaces, NumberVertices equal to oMesh's property values. Declare the mesh with Position, Normal and Textured as the vertex format. We'll call this mesh object oFinal.
5) Lock the Vertex Buffers of both mesh objects
6) Loop through the arrays of data results from locking the Vertex Buffers setting the vertex of oFinal equal to the data of oMesh. And then including your Texture data.
7) Once done looping, Unlock both vertex buffers
8) Lock the index buffer of oMesh to get the index array
9) Call oFinal's IndexBuffer's SetData method to set the index buffer's data
10) Dispose and release oMesh, return oFinal

Then, in your render loop, render oMesh normally. You don't have to setFVF or anything. This should work. Clear as mud?

Share this post


Link to post
Share on other sites
Okay, yeah, changing to face- instead of vertex normals worked. I didn't have to calculate them, there's a lib3ds function for doing that, too. I'm actually happy with this, I guess it was with the way lib3ds gathers its adjacency data and does the vertex normal calculations, probably due to the difference between left- and right-handedness.

A next step would be to go back to the, for most cases better looking, vertex normals and actually implement that awful conversion properly.

Thanks for all those who replied and helped me resolve this.

Share this post


Link to post
Share on other sites
Quote:
Original post by Buckeye
[snip...]
Also, you may need to "unwind" the vertices (which, from your data above, appear to be CCW rather than CW) either by changing the order of the vertices for each face, or changing the indices for each triangle to render then CW (which DirectX expects). If you unwind the vertices, do that before you compute normals.


Normal winding order for DX is CCW, for gl its CW.

All interpolated values, values you pass from vertex to pixel shader, should be renormalized. The interpolation doesn't garantee that a vector stays normalized.

You can safely multiply a float4x4 and a float3 only the left upper 3x3 part will be used for the transformation.

Next what you want to do is proper per pixel lighting by actually calculating the direction to the light in the vertex shader. Be aware that you will have to output your vertex position twice as you can't read from POSITION in the pixel shader.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this