Sign in to follow this  
dave

Normal Mapping With Deferred Rendering

Recommended Posts

I understand how normal mapping works with forward rendering and i think i know how i am supposed to do it in deferred rendering, however i cant find any decent articles. If you can please check out the source code in the final post of this thread, as it is what i followed roughly. This appears wrong. It looks like he is transforming a tangent space normal into tangent space with that TBN matrix. If i am correct then the question becomes, how do i do this in deferred rendering where the normal g-buffer normals must be in world space? I have been looking into converting the inverse transpose of the TBN matrix but there is no inverse intrinsic to do so, so i am a bit stumped. My object pass vertex shader:
VS_OUTPUT vs_main( VS_INPUT In )
{
	VS_OUTPUT Out;

	Out.Pos						= mul(float4(In.Pos,1.0), matWorldViewProjection);
	Out.PosInWorld				= mul(float4(In.Pos,1.0), matWorld);
	Out.Normal					= mul(float4(In.Normal,1.0), matWorld);
	Out.TangentWorldMatrix[0]	= mul(float4(In.Tangent,1.0f), matWorld).xyz;
	Out.TangentWorldMatrix[1]	= mul(float4(In.Binormal,1.0f), matWorld).xyz;
	Out.TangentWorldMatrix[2]	= mul(float4(In.Normal,1.0f), matWorld).xyz;
	Out.Tex						= In.Tex;
	
	return Out;
}
My object pass pixel shader:
PS_OUTPUT ps_main( VS_OUTPUT In )
{
	PS_OUTPUT Out;
	
	float4		SampledNormal = tex2D(NormalMapSampler, In.Tex);
	SampledNormal = normalize(2.0f * (SampledNormal - 0.5f));
	
	float3 WorldNormal = normalize(mul(SampledNormal, In.TangentWorldMatrix));
	
	Out.PosInWorld		= float4(In.PosInWorld,1.0f);
	
	if ( HasNormalMap == true )
	{
		Out.NormalInWorld	= float4(WorldNormal,1.0f);
	}
	else
	{
		Out.NormalInWorld	= float4(In.Normal,1.0f);
	}
	
	Out.Albedo			= tex2D(TextureMapSampler, In.Tex);
   
	return Out;
}
Another thing i have stumbled accross just now is the hint that i should be filling the normal g buffer with view space normals and doing lighting in the shade pass in view space. Should this be the case? Any help appreciated, thanks. [Edited by - Dave on July 20, 2009 2:43:41 PM]

Share this post


Link to post
Share on other sites
Hi,

Your shader code looks correct but I've 3 questions:
* Does it work for flat surfaces (i.e. when HasNormalMap is false)?
* Is your vertex declaration for bumped surfaces correct? Did you clone your mesh with TBN basis vertex declaration?
* Why don't you pack your normal data? Look at the snippet below:

PS_DEFERRED_OUT PS_Deferred (VS_OUTPUT vsIn)
{
PS_DEFERRED_OUT psOut = (PS_DEFERRED_OUT)0;

//...
//Do your normal calculations (with TBN basis or not) and obtain psOut.NormalInWorld
//...
psOut.NormalInWorld = 0.5 * (1 + psOut.NormalInWorld);

//...
return psOut;

//In deferred pass, you can obtain real world-space normal with this formula:
// float3 n = 2 * SampledNormalFromGBuffer - 1.0f;
}


Quote:
Another thing i have stumbled accross just now is the hint that i should be filling the normal g buffer with view space normals and doing lighting in the shade pass in view space. Should this be the case?

Filling G-Buffer with View-Space normal vectors is meaningful if you want to reconstruct normal.z. I mean, if you need an extra field in your GBuffer, you can store ViewSpaceNormal.xy and you can reconstruct ViewSpaceNormal.z by using ViewSpaceNormal.z = sqrt(1 - dot (ViewSpaceNormal.xy, ViewSpaceNormal.xy)); formula in deferred pass.

Hope this helps.
Rohat.

Share this post


Link to post
Share on other sites
* Does it work for flat surfaces (i.e. when HasNormalMap is false)?

Everything looks fine with the normals straight from the models, applied to the g buffer in world space.

* Is your vertex declaration for bumped surfaces correct? Did you clone your mesh with TBN basis vertex declaration?

I run this accross all models passed in. It appears to generate information and doesn't return any errors, nor are any passed back from d3d.

void	ModelManager::GenerateTangents( Model* pNewModel )
{
D3DXMesh pMesh( pNewModel->GetMesh() );

D3DVERTEXELEMENT9 EndElement = {0xFF,0,D3DDECLTYPE_UNUSED,0,0,0};

D3DVERTEXELEMENT9 CurrentDeclaration[MAX_FVF_DECL_SIZE];

pMesh->GetDeclaration(&CurrentDeclaration[0]);

// Find the end of the declaration and insert space for tangents.
UInt32 EndIndex( 0 );

for ( UInt32 Index( 0 ); Index < MAX_FVF_DECL_SIZE; ++Index )
{
if ( CurrentDeclaration[ Index ].Type == D3DDECLTYPE_UNUSED )
{
EndIndex = Index;
break;
}
}

CurrentDeclaration[ EndIndex + 2 ] = EndElement;

D3DVERTEXELEMENT9& rTangentElement( CurrentDeclaration[EndIndex] );
D3DVERTEXELEMENT9& rBiNormalElement( CurrentDeclaration[EndIndex+1] );
D3DVERTEXELEMENT9& rPreviousElement( CurrentDeclaration[EndIndex-1] );

rTangentElement.Offset = rPreviousElement.Offset + 12;
rTangentElement.Method = D3DDECLMETHOD_DEFAULT;
rTangentElement.Stream = rPreviousElement.Stream;
rTangentElement.Type = D3DDECLTYPE_FLOAT3;
rTangentElement.Usage = D3DDECLUSAGE_TANGENT;
rTangentElement.UsageIndex = 0;

rBiNormalElement.Offset = rTangentElement.Offset + 12;
rBiNormalElement.Method = D3DDECLMETHOD_DEFAULT;
rBiNormalElement.Stream = rPreviousElement.Stream;
rBiNormalElement.Type = D3DDECLTYPE_FLOAT3;
rBiNormalElement.Usage = D3DDECLUSAGE_BINORMAL;
rBiNormalElement.UsageIndex = 0;

D3DXMesh NewMesh;

HRESULT Result = pMesh->CloneMesh( D3DXMESH_VB_MANAGED | D3DXMESH_IB_MANAGED, &CurrentDeclaration[0], m_Device, &NewMesh );

pNewModel->SetMesh( NewMesh );

DWORD* pCharBuffer( new DWORD[3 * NewMesh->GetNumFaces()] );
NewMesh->GenerateAdjacency( 0.001f, pCharBuffer );


D3DXCleanMesh( D3DXCLEAN_BACKFACING, NewMesh, pCharBuffer, &NewMesh, pCharBuffer, NULL );

static UInt32 TexStage(0);
static UInt32 TangentStage(0);
static UInt32 BinormalStage(0);

Result = D3DXComputeTangent( pNewModel->GetMesh(), TexStage, TangentStage, BinormalStage, TRUE, pCharBuffer );

if ( Result == D3DERR_INVALIDCALL )
Result = Result;
else if ( Result == D3DXERR_INVALIDDATA )
Result = Result;
}


* Why don't you pack your normal data? Look at the snippet below:

I am not ready to optimise yet, but thanks.

Share this post


Link to post
Share on other sites
A TBN matrix actually transforms from tangent-space to object-space...not the other way around. Typically with forward rendering you take your TBN matrix and rotate it by your world matrix (so that your TBN matrix will now transform from tangent-space to world-space) and then transform your light direction and eye direction by the transpose of the TBN matrix. The shader code usually looks like this in samples:


tbnMatrix = float3x3(mul(In.Tangent, WorldMatrix), mul(In.Binormal, WorldMatrix), mul(In.Normal, WorldMatrix));
Out.EyeDirection = mul(tbnMatrix, eyeDirection);
Out.LightDirection = mul(tbnMatrix, lightDirection);


The part that most people miss (and most people never mention in their sample code) is that when calcluation the Eye/Light directions the order of the vector and the matrix is reversed from what you normally do (usually you do mul(vector, matrix). This is ultimately the same as taking the transpose of a matrix and doing the transformation in the normal order, like this:


tangentToWorld = float3x3(mul(In.Tangent, WorldMatrix), mul(In.Binormal, WorldMatrix), mul(In.Normal, WorldMatrix));
worldToTangent = transpose(tangentToWorld);
Out.EyeDirection = mul(eyeDirection , worldToTangent);
Out.LightDirection = mul(lightDirection , worldToTangent);


The reason taking the transpose works is that for an orthogonal transformation matrix, the transpose is equal to its inverse. In most cases your tangent basis will be orthogonal...or at least close enough to it.

As far as using view-space instead of world-space...in most cases it's used because it can be easier or cheaper to reconstruct view-space position from depth, and it allows you to simply some parts of the lighting calculations. It doesn't really matter too much, as long as you keep track of what coordinate space you're in and make sure that everything matches.

Share this post


Link to post
Share on other sites
Quote:
Original post by programci_84

Filling G-Buffer with View-Space normal vectors is meaningful if you want to reconstruct normal.z. I mean, if you need an extra field in your GBuffer, you can store ViewSpaceNormal.xy and you can reconstruct ViewSpaceNormal.z by using ViewSpaceNormal.z = sqrt(1 - dot (ViewSpaceNormal.xy, ViewSpaceNormal.xy)); formula in deferred pass.


Be careful with that. You can't assume that the sign of z is always negative in view-space (or positive, if you're using right-handed coordinates) when you're using perspective projection or normal maps. Insomniac has some pictures showing the errors you can get here.

Share this post


Link to post
Share on other sites
Ok so here is the ultimate screenshot of doom:


As you can see the normal mapping isn't quite right. Looks like the tangent space normal isn't being converted to world space correctly. The demo of this is it rotating and its clear that something is wrong because the normal map on each side of the cube is different.

MJP, i'm sure the answer is in your post somewhere :D.

Share this post


Link to post
Share on other sites
Crap! I actually saw the problem before in your code, and completely forgot to mention it by the time I was done rambling. This part right here:


Out.Normal = mul(float4(In.Normal,1.0), matWorld);
Out.TangentWorldMatrix[0] = mul(float4(In.Tangent,1.0f), matWorld).xyz;
Out.TangentWorldMatrix[1] = mul(float4(In.Binormal,1.0f), matWorld).xyz;
Out.TangentWorldMatrix[2] = mul(float4(In.Normal,1.0f), matWorld).xyz;


You don't want your w component to be 1.0 when you're transforming your Normal, Binormal, or Tangent (or any normalized direction vector, for that matter). The reason why is because if w is 1.0, then transforming it by a 4x4 matrix will cause it to be translated by the matrix as well. However you don't want to translate direction vectors, you just want to rotate them. Try this instead:


Out.Normal = mul(In.Normal, matWorld);
Out.TangentWorldMatrix[0] = mul(In.Tangent, matWorld).xyz;
Out.TangentWorldMatrix[1] = mul(In.Binormal, matWorld).xyz;
Out.TangentWorldMatrix[2] = Out.Normal;


Share this post


Link to post
Share on other sites
Hey,

I'm afraid that didn't appear to have any affect at all. Here is the updated shader:
VS_OUTPUT vs_main( VS_INPUT In )
{
VS_OUTPUT Out;

Out.Pos = mul(float4(In.Pos,1.0), matWorldViewProjection);
Out.PosInWorld = mul(float4(In.Pos,1.0), matWorld);
Out.Normal = mul(In.Normal, matWorld);
Out.TangentWorldMatrix[0] = mul(In.Tangent, matWorld).xyz;
Out.TangentWorldMatrix[1] = mul(In.Binormal, matWorld).xyz;
Out.TangentWorldMatrix[2] = Out.Normal;
Out.Tex = In.Tex;

return Out;
}

PS_OUTPUT ps_main( VS_OUTPUT In )
{
PS_OUTPUT Out;

float4 SampledNormal = tex2D(NormalMapSampler, In.Tex);
SampledNormal = normalize(2.0f * (SampledNormal - 0.5f));

float3 WorldNormal = normalize(mul(SampledNormal, In.TangentWorldMatrix));

Out.PosInWorld = float4(In.PosInWorld,1.0f);
if ( HasNormalMap == true )
{
Out.NormalInWorld = float4(WorldNormal,1.0f);
}
else
{
Out.NormalInWorld = float4(In.Normal,1.0f);
}
Out.Albedo = tex2D(TextureMapSampler, In.Tex);

return Out;
}



Thanks for the help,

Share this post


Link to post
Share on other sites
Quote:
Original post by MJP
Hmmm...the only other issue I see with your code is that you're not normalizing your Normal/Binormal/Tangent after interpolation before using it to transform the normal sampled from the normal map.


I could be wrong, but that wouldn't account for the wrong 'direction' look for the normal map would it?

Share this post


Link to post
Share on other sites
Quote:
Original post by MJP
A TBN matrix actually transforms from tangent-space to object-space...not the other way around.


You sure? All the shaders I've written use the TBN matrix to transform from object space to tangent space, and the transpose of it to transform from tangent space to object space.

Dave, transpose your TBN matrix. In fact, what I do, is this:

// vert


OUT.n = IN.n;
OUT.t = IN.t;
OUT.b = cross(IN.t, IN.n);



// frag

/* fetch normalTexel */
normalTexel.xyz = normalize(normalTexel.xyz * 2 - 1);

float3x3 invtbn = transpose(float3x3(IN.t, IN.b, IN.n));
float3 normal = mul(invtbn, normalTexel.xyz); // to object space

// transform from object space to view-space
normal = normalize(mul((float3x3)iTWVMat, normal));


Note that not only am I using view-space (which you don't have to), but I'm also multiplying by the inverse-transpose of the world-view matrix; this is done so that non-uniformly scaled meshes get proper normals, which would end up distorted if you used the world(-view) matrix directly.

Share this post


Link to post
Share on other sites
Quote:
Original post by Dave
Quote:
Original post by MJP
Hmmm...the only other issue I see with your code is that you're not normalizing your Normal/Binormal/Tangent after interpolation before using it to transform the normal sampled from the normal map.


I could be wrong, but that wouldn't account for the wrong 'direction' look for the normal map would it?


Yeah that would usually just cause minor errors.

At this point I'm not really sure what the problem is. You might have to step through all the way in PIX (in both your G-Buffer shader and Lighting shader) to make sure the values look like what you'd expect them to be.

Share this post


Link to post
Share on other sites
Quote:
Original post by nullsquared
Quote:
Original post by MJP
A TBN matrix actually transforms from tangent-space to object-space...not the other way around.


You sure? All the shaders I've written use the TBN matrix to transform from object space to tangent space, and the transpose of it to transform from tangent space to object space.


Are you sure you're not just constructing the TBN matrix already transposed?

Share this post


Link to post
Share on other sites
Interesting...

I have just run a quad model laying on the xy plane through the D3DXComputeTangent() function, no errors or warnings returned by D3D. I locked the vertex buffer and the stride of the vertex seems ok (it looks like d3d has padded the float2 texture coordinates to a float3) and for where the tangents and normals should be appear to be (0,0,0).

Here is the cloning and tangent computation code:
	D3DXMesh	pMesh( pNewModel->GetMesh() );

D3DVERTEXELEMENT9 EndElement = {0xFF,0,D3DDECLTYPE_UNUSED,0,0,0};

D3DVERTEXELEMENT9 CurrentDeclaration[MAX_FVF_DECL_SIZE];

pMesh->GetDeclaration(&CurrentDeclaration[0]);

// Find the end of the declaration and insert space for tangents.
UInt32 EndIndex( 0 );

for ( UInt32 Index( 0 ); Index < MAX_FVF_DECL_SIZE; ++Index )
{
if ( CurrentDeclaration[ Index ].Type == D3DDECLTYPE_UNUSED )
{
EndIndex = Index;
break;
}
}

CurrentDeclaration[ EndIndex + 2 ] = EndElement;

D3DVERTEXELEMENT9& rTangentElement( CurrentDeclaration[EndIndex] );
D3DVERTEXELEMENT9& rBiNormalElement( CurrentDeclaration[EndIndex+1] );
D3DVERTEXELEMENT9& rPreviousElement( CurrentDeclaration[EndIndex-1] );

rTangentElement.Offset = rPreviousElement.Offset + 12;
rTangentElement.Method = D3DDECLMETHOD_DEFAULT;
rTangentElement.Stream = rPreviousElement.Stream;
rTangentElement.Type = D3DDECLTYPE_FLOAT3;
rTangentElement.Usage = D3DDECLUSAGE_TANGENT;
rTangentElement.UsageIndex = 0;

rBiNormalElement.Offset = rTangentElement.Offset + 12;
rBiNormalElement.Method = D3DDECLMETHOD_DEFAULT;
rBiNormalElement.Stream = rPreviousElement.Stream;
rBiNormalElement.Type = D3DDECLTYPE_FLOAT3;
rBiNormalElement.Usage = D3DDECLUSAGE_BINORMAL;
rBiNormalElement.UsageIndex = 0;

D3DXMesh NewMesh;

HRESULT Result = pMesh->CloneMesh( D3DXMESH_VB_MANAGED | D3DXMESH_IB_MANAGED, &CurrentDeclaration[0], m_Device, &NewMesh );

pNewModel->SetMesh( NewMesh );

DWORD* pCharBuffer( new DWORD[3 * NewMesh->GetNumFaces()] );
NewMesh->GenerateAdjacency( 0.001f, pCharBuffer );


D3DXCleanMesh( D3DXCLEAN_BOWTIES , NewMesh, pCharBuffer, &NewMesh, pCharBuffer, NULL );
D3DXCleanMesh( D3DXCLEAN_BACKFACING , NewMesh, pCharBuffer, &NewMesh, pCharBuffer, NULL );

static UInt32 TexStage(0);
static UInt32 TangentStage(0);
static UInt32 BinormalStage(0);

Result = D3DXComputeTangent( pNewModel->GetMesh(), TexStage, TangentStage, BinormalStage, TRUE, pCharBuffer );

if ( Result == D3DERR_INVALIDCALL )
Result = Result;
else if ( Result == D3DXERR_INVALIDDATA )
Result = Result;

void* pData( NULL );
NewMesh->LockVertexBuffer( 0, &pData );

if ( pData != NULL )
{

}

NewMesh->UnlockVertexBuffer();




Here is the quad x file as exported from Blender:

xof 0303txt 0032


template VertexDuplicationIndices {
<b8d65549-d7c9-4995-89cf-53a9a8b031e3>
DWORD nIndices;
DWORD nOriginalVertices;
array DWORD indices[nIndices];
}
template XSkinMeshHeader {
<3cf169ce-ff7c-44ab-93c0-f78f62d172e2>
WORD nMaxSkinWeightsPerVertex;
WORD nMaxSkinWeightsPerFace;
WORD nBones;
}
template SkinWeights {
<6f0d123b-bad2-4167-a0d0-80224f25fabb>
STRING transformNodeName;
DWORD nWeights;
array DWORD vertexIndices[nWeights];
array float weights[nWeights];
Matrix4x4 matrixOffset;
}

Frame RootFrame {

FrameTransformMatrix {
1.000000,0.000000,0.000000,0.000000,
0.000000,-0.000000,1.000000,0.000000,
0.000000,1.000000,0.000000,0.000000,
0.000000,0.000000,0.000000,1.000000;;
}
Frame Plane {

FrameTransformMatrix {
1.000000,0.000000,0.000000,0.000000,
0.000000,-0.000000,1.000000,0.000000,
0.000000,-1.000000,-0.000000,0.000000,
0.000000,0.000000,0.000000,1.000000;;
}
Mesh {
4;
1.000000; 1.000000; 0.000000;,
-1.000000; 1.000000; 0.000000;,
-1.000000; -1.000000; 0.000000;,
1.000000; -1.000000; 0.000000;;
1;
4; 0, 3, 2, 1;;
MeshMaterialList {
0;
1;
0;;
} //End of MeshMaterialList
MeshNormals {
4;
0.000000; 0.000000; 1.000000;,
0.000000; 0.000000; 1.000000;,
0.000000; 0.000000; 1.000000;,
0.000000; 0.000000; 1.000000;;
1;
4; 0, 3, 2, 1;;
} //End of MeshNormals
MeshTextureCoords {
4;
1.000000;-1.000000;,
0.000000;-1.000000;,
0.000000;-0.000000;,
1.000000;-0.000000;;
} //End of MeshTextureCoords
}
}
}




I don't see anything wrong with the model or any reason for D3DXComputeTangent to fail to create them despite returning S_OK.

Share this post


Link to post
Share on other sites
The vertex declaration for the clones mesh was wrong. If you look at the vertex element for the texture coordinates its down as 12 bytes long instead of 8. This threw off the tangent calculations etc.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this