Jump to content
  • Advertisement
  • entries
    338
  • comments
    2045
  • views
    1170866

Normal mapping / generating tangents for skinned m

Sign in to follow this  
dgreen02

645 views

*stretches out* Ok this is gonna be a long one....

Hopefully this will be the final graphical effect I'm going to add into the game, and it looks like it'll be a little while before I'm happy with it.

I've tried to add this into the game previously, each time getting closer, but utlimatly getting caught up on something.

First off, I'm forced to use a ooooold version of the DXSDK like 4 years old because any new versions fail to load almost all of my .X files [I have ~500 of them associated with the game]. I've spent days trying to get that issue resolved in past, but I'm just going to tough it out for this game. Unfortunatly this also means I'm limited to PS2.0.

The other issue is that D3DXComputeTangent() fails for all of my skinned meshes, and I've spent days previously trying to work out that issue. Because the verison of the SDK I use doesn't even have the D3DXComputeTangentFrameEx() function.

So I decided I have to generate the tangents by hand. After a few days of coding / debugging issues I FINALLY have this working in my engine.

Google searchs for "skinned mesh tangents" returns nearly nothing, and I couldn't find any code/examples showing how to generate the tangents by hand for a skinned D3DXMESH. I would have been happy with, any functional code that just uses the D3DXTangent() functions with skinned meshes.

Well here you go google - This code should be pluggable into the SkinnedMesh class in the DXSDK, at least the version I have. No promises about any part of this [I wrote it last night]. I just hope somebody will be helped out someday by this code...I couldn't find sh*t when I was googling around for code/etc. Also, the debug statments are obviously specifc to my game, so you gotta modify/take those out.


void CSkinnedMesh::GenerateTangents(LPD3DXFRAME pFrameBase)
{
//DWORD NewFVF = (pMeshContainer->MeshData.pMesh->GetFVF() & D3DFVF_POSITION_MASK) | D3DFVF_NORMAL | D3DFVF_TEX2 | D3DFVF_LASTBETA_UBYTE4 | D3DFVF_TEXCOORDSIZE2(0) | D3DFVF_TEXCOORDSIZE4(1);
D3DXFRAME_DERIVED *pFrame = (D3DXFRAME_DERIVED*)pFrameBase;
D3DXMESHCONTAINER_DERIVED *pMeshContainer;
pMeshContainer = (D3DXMESHCONTAINER_DERIVED *)pFrame->pMeshContainer;

while(pMeshContainer != NULL)
{
int NumVerts = pMeshContainer->MeshData.pMesh->GetNumVertices();
int NumFaces = pMeshContainer->MeshData.pMesh->GetNumFaces();
int NumBytesPerVertex = pMeshContainer->MeshData.pMesh->GetNumBytesPerVertex();
bool MeshUses32BitIndices = (pMeshContainer->MeshData.pMesh->GetOptions() & D3DXMESH_32BIT);
int NumBytesPerIndex = (MeshUses32BitIndices ? 4:2);

//Output mesh information
GameEngine.LogEntry("Generating Tangents %s - [NumFaces %d | NumBytesPerVert %d | NumBytesPerIndex %d | NumVerts %d | sizeof(MeshVertex) = %d]", pFrame->Name, NumFaces, NumBytesPerVertex, NumBytesPerIndex, NumVerts, sizeof(SkinnedMeshVertex));

//Check to make sure sizes of verts match up
if(sizeof(SkinnedMeshVertex) != NumBytesPerVertex)
{
GameEngine.LogEntry("WARNING! In GenerateTangents() [MeshVertex size (%d) != NumBytesPerVertex (%d)]", sizeof(SkinnedMeshVertex), NumBytesPerVertex);
} //end of if
if(sizeof(SkinnedMeshVertex) < NumBytesPerVertex) return;

BYTE* pVertices = NULL;
WORD* pIndices = NULL;
long int ReadPosition = 0;

//Allocate arrays to hold verts + faces
SkinnedMeshVertex *pMeshVerts = new SkinnedMeshVertex[NumVerts];
SkinnedMeshTangentTriangle *pTangentTris = new SkinnedMeshTangentTriangle[NumFaces];

//Lock the mesh's VB
HRESULT hr = pMeshContainer->MeshData.pMesh->LockVertexBuffer(/*D3DLOCK_READONLY*/0, (LPVOID*)&pVertices);

if(!FAILED(hr))
{
//Copy the meshs' vertex data into the temp array using one memcpy
//memcpy(pMeshVerts, pVertices, NumBytesPerVertex*NumVerts);

//Loop through each vertex moving it into the temp array, and output it as well...
for(int i = 0; i < NumVerts; i++)
{
//Copy the vertex from mesh to temp memory...
memcpy(&pMeshVerts, pVertices+ReadPosition, NumBytesPerVertex);

//Output vertex for debugging purposes...
/* GameEngine.LogEntry(" Vert %d - [Position %0.3f %0.3f %0.3f] [Normal %0.3f %0.3f %0.3f] [TexCoord %0.2f %0.2f] [Tangent %0.2f %0.2f %0.2f %0.2f] [BlendWeights %0.2f %0.2f %0.2f] [BlendIndices %d %d %d %d]", i,
pMeshVerts.Position.x, pMeshVerts.Position.y, pMeshVerts.Position.z,
pMeshVerts.Normal.x, pMeshVerts.Normal.y, pMeshVerts.Normal.z,
pMeshVerts.TexCoord.x, pMeshVerts.TexCoord.y,
pMeshVerts.Tangent.x, pMeshVerts.Tangent.y, pMeshVerts.Tangent.z, pMeshVerts.Tangent.w,
pMeshVerts.BlendWeights.x, pMeshVerts.BlendWeights.y, pMeshVerts.BlendWeights.z,
pMeshVerts.BlendIndices[0], pMeshVerts.BlendIndices[1], pMeshVerts.BlendIndices[2], pMeshVerts.BlendIndices[3]);
*/

ReadPosition+=NumBytesPerVertex;
} //end of for loop

//Now generate triangle array from the mesh's index buffer...
//First lock the index buffer
pMeshContainer->MeshData.pMesh->LockIndexBuffer(0, (LPVOID*)&pIndices);

//For each face get the indices of the 3 verts that make up that face.
for(i = 0; i < NumFaces; i++)
{
for(int j = 0; j < 3; j++)
{
if(MeshUses32BitIndices)
{
//32-bit indices
} //end of if
else
{
//16-bit indices
WORD tempIndex;
memcpy(&tempIndex, pIndices, NumBytesPerIndex);
pTangentTris.Index[j] = tempIndex;
} //end of else
pIndices++;
} //end of for loop
//GameEngine.LogEntry("Tri %d - [%d %d %d]", i, pTangentTris.Index[0], pTangentTris.Index[1], pTangentTris.Index[2]);
} //end of for loop

//Unlock index buffer
pMeshContainer->MeshData.pMesh->UnlockVertexBuffer();

//Actually compute tangents inside this function.
//The array of verts will be filled with correct tangent data after this fucntion.
CalculateTangentArrayMeshVerts(NumVerts, NumFaces, pMeshVerts, pTangentTris);

//Loop through each vertex and move the data [including the tangents] back into the mesh...
//Could use one memcpy call, but doing it this way allows other per-vertex calculations to be done here.
ReadPosition = 0;
for(i = 0; i < NumVerts; i++)
{
//pMeshVerts.TexCoord.x = 0;
//pMeshVerts.TexCoord.y = 0;

//Output vertex for debugging purposes...
/* GameEngine.LogEntry(" Vert %d - [Position %0.3f %0.3f %0.3f] [Normal %0.3f %0.3f %0.3f] [TexCoord %0.2f %0.2f] [Tangent %0.2f %0.2f %0.2f %0.2f] [BlendWeights %0.2f %0.2f %0.2f] [BlendIndices %d %d %d %d]", i,
pMeshVerts.Position.x, pMeshVerts.Position.y, pMeshVerts.Position.z,
pMeshVerts.Normal.x, pMeshVerts.Normal.y, pMeshVerts.Normal.z,
pMeshVerts.TexCoord.x, pMeshVerts.TexCoord.y,
pMeshVerts.Tangent.x, pMeshVerts.Tangent.y, pMeshVerts.Tangent.z, pMeshVerts.Tangent.w,
pMeshVerts.BlendWeights.x, pMeshVerts.BlendWeights.y, pMeshVerts.BlendWeights.z,
pMeshVerts.BlendIndices[0], pMeshVerts.BlendIndices[1], pMeshVerts.BlendIndices[2], pMeshVerts.BlendIndices[3]);*/

//Copy vertex data back into mesh...
memcpy(pVertices+ReadPosition, &pMeshVerts, NumBytesPerVertex);

ReadPosition+=NumBytesPerVertex;
} //end of for loop

// Finished so unlock VBs
pMeshContainer->MeshData.pMesh->UnlockVertexBuffer();

//Delete memory used to store verts/faces
delete[] pMeshVerts;
delete[] pTangentTris;
} //end of if
else
{
GameEngine.LogEntry("LockVertexBuffer() #1 FAILED!");
} //end of else

pMeshContainer = (D3DXMESHCONTAINER_DERIVED *)pMeshContainer->pNextMeshContainer;
} //end of while loop

if(pFrame->pFrameSibling != NULL)
{
GenerateTangents(pFrame->pFrameSibling);
} //end of if

if(pFrame->pFrameFirstChild != NULL)
{
GenerateTangents(pFrame->pFrameFirstChild);
} //end of if
} //end of GenerateTangents function

//This function is a slight modification of the one found @ http://www.terathon.com/code/tangent.php [ by Eric Lengyel ]
//Now uses D3DXVECTOR3 & and the SkinnedMeshVertex / SkinnedMeshTangentTriangle structs which represent
//the FVF / Decl of the mesh. Also checks for divide by zero & other error checking.
void CalculateTangentArrayMeshVerts(unsigned long VertexCount, unsigned long TriangleCount, SkinnedMeshVertex *pVerts, const SkinnedMeshTangentTriangle *pTriangles)
{
//The code generates a four-component tangent T in which the handedness of
//the local coordinate system is stored as +-1 in the w-coordinate. The bitangent
//vector B is then given by B = (N x T)Tw.

//For mathematical details, see http://www.terathon.com/books/mathgames2.html

D3DXVECTOR3 *tan1 = new D3DXVECTOR3[VertexCount * 2];
D3DXVECTOR3 *tan2 = tan1 + VertexCount;

//Clear the memory
memset(tan1, 0, sizeof(D3DXVECTOR3) * VertexCount * 2);

for(long a = 0; a < TriangleCount; a++)
{
unsigned short i1 = pTriangles->Index[0];
unsigned short i2 = pTriangles->Index[1];
unsigned short i3 = pTriangles->Index[2];

//Do bounds checking making sure each index is between 0 & VertexCount.
if(i1 >= VertexCount || i1 < 0 || i2 >= VertexCount || i2 < 0 || i3 >= VertexCount || i3 < 0)
{
GameEngine.LogEntry("CalculateTangentArrayMeshVerts() ERROR! Bounds Exceeded! [%d %d %d]", i1, i2, i3);
return;
} //end of if

const D3DXVECTOR3& v1 = pVerts[i1].Position;
const D3DXVECTOR3& v2 = pVerts[i2].Position;
const D3DXVECTOR3& v3 = pVerts[i3].Position;

const D3DXVECTOR2& w1 = pVerts[i1].TexCoord;
const D3DXVECTOR2& w2 = pVerts[i2].TexCoord;
const D3DXVECTOR2& w3 = pVerts[i3].TexCoord;

double x1 = v2.x - v1.x;
double x2 = v3.x - v1.x;
double y1 = v2.y - v1.y;
double y2 = v3.y - v1.y;
double z1 = v2.z - v1.z;
double z2 = v3.z - v1.z;

double s1 = w2.x - w1.x;
double s2 = w3.x - w1.x;
double t1 = w2.y - w1.y;
double t2 = w3.y - w1.y;

double DivValue = (s1 * t2 - s2 * t1);

//Check for divide by zero...
if(DivValue == 0.0f)
{
GameEngine.LogEntry("CalculateTangentArrayMeshVerts() WARNING! Divide by 0 detected! Indices = [%d %d %d] TexCoords = [%f %f] [%f %f] [%f %f]", i1, i2, i3, w1.x, w1.y, w2.x, w2.y, w3.x, w3.y);
DivValue = 1.0F;
} //end of if

double r = 1.0F / DivValue;
D3DXVECTOR3 sdir = D3DXVECTOR3((t2 * x1 - t1 * x2) * r, (t2 * y1 - t1 * y2) * r, (t2 * z1 - t1 * z2) * r);
D3DXVECTOR3 tdir = D3DXVECTOR3((s1 * x2 - s2 * x1) * r, (s1 * y2 - s2 * y1) * r, (s1 * z2 - s2 * z1) * r);

tan1[i1] += sdir;
tan1[i2] += sdir;
tan1[i3] += sdir;

tan2[i1] += tdir;
tan2[i2] += tdir;
tan2[i3] += tdir;

pTriangles++;
} //end of for loop

for(long a = 0; a < VertexCount; a++)
{
const D3DXVECTOR3& n = pVerts[a].Normal;
const D3DXVECTOR3& t = tan1[a];

// Gram-Schmidt orthogonalize
D3DXVECTOR3 TempTangent = (t - n * D3DXVec3Dot(&n, &t));
D3DXVec3Normalize(&TempTangent, &TempTangent);

// Calculate handedness & set final tangent
pVerts[a].Tangent.x = TempTangent.x;
pVerts[a].Tangent.y = TempTangent.y;
pVerts[a].Tangent.z = TempTangent.z;
pVerts[a].Tangent.w = (D3DXVec3Dot(D3DXVec3Cross(&TempTangent, &n, &t), &tan2[a]) < 0.0F) ? -1.0F : 1.0F;
// GameEngine.LogEntry("%d = %f %f %f %f", a, pVerts[a].Tangent.x, pVerts[a].Tangent.y, pVerts[a].Tangent.z, pVerts[a].Tangent.w);
} //end of for loop

delete[] tan1;
} //end of CalculateTangentArrayMeshVerts function







As noted in the comments, the actual code that calculates the tangents is slightly modified from http://www.terathon.com/code/tangent.php [by Eric Lengyel, props to him].

Here are the structs used in the above code...

struct SkinnedMeshVertex
{
D3DXVECTOR3 Position;
D3DXVECTOR3 BlendWeights;
unsigned char BlendIndices[4];
D3DXVECTOR3 Normal;
D3DXVECTOR2 TexCoord;
D3DXVECTOR4 Tangent;
};

struct SkinnedMeshTangentTriangle
{
unsigned short Index[3];
};








You just load the skinned mesh as usual, then use this fucntion to make room for the tangents inside the mesh's vertex decl.

void CSkinnedMesh::ForceDeclRecursive(LPD3DXFRAME pFrameBase)
{
D3DXFRAME_DERIVED *pFrame = (D3DXFRAME_DERIVED*)pFrameBase;
D3DXMESHCONTAINER_DERIVED *pMeshContainer;
HRESULT hr;

pMeshContainer = (D3DXMESHCONTAINER_DERIVED *)pFrame->pMeshContainer;

while(pMeshContainer != NULL)
{
//Force Decl...
//NumBytesPerVert = 64
//Element #0 - D3DDECLUSAGE_POSITION | Offset 0 | Type 2 | Usage Index 0
//Element #1 - D3DDECLUSAGE_BLENDWEIGHT | Offset 12 | Type 2 | Usage Index 0
//Element #2 - D3DDECLUSAGE_BLENDINDICES | Offset 24 | Type 4 | Usage Index 0
//Element #3 - D3DDECLUSAGE_NORMAL | Offset 28 | Type 2 | Usage Index 0
//Element #4 - D3DDECLUSAGE_TEXCOORD | Offset 40 | Type 1 | Usage Index 0
//Element #5 - D3DDECLUSAGE_TEXCOORD | Offset 48 | Type 3 | Usage Index 1

D3DVERTEXELEMENT9 NewVertexDecl[] =
{
{ 0, 0, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_POSITION, 0 },
{ 0, 12, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_BLENDWEIGHT, 0 },
{ 0, 24, D3DDECLTYPE_D3DCOLOR, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_BLENDINDICES, 0 },
{ 0, 28, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_NORMAL, 0 },
{ 0, 40, D3DDECLTYPE_FLOAT2, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_TEXCOORD, 0 },
{ 0, 48, D3DDECLTYPE_FLOAT4, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_TEXCOORD, 1 },

D3DDECL_END()
};

LPD3DXMESH pMesh2;
hr = pMeshContainer->MeshData.pMesh->CloneMesh(pMeshContainer->MeshData.pMesh->GetOptions(), NewVertexDecl, D3DDEVICE, &pMesh2);
if(!FAILED(hr))
{
pMeshContainer->MeshData.pMesh->Release();
pMeshContainer->MeshData.pMesh = pMesh2;
pMesh2 = NULL;
} //end of if
else
{
GameEngine.LogEntry("CSkinnedMesh::[CloneMesh() failed in ForceDeclRecursive()]");
return;
} //end of eles

hr = pMeshContainer->MeshData.pMesh->UpdateSemantics(NewVertexDecl);
if(FAILED(hr))
{
GameEngine.LogEntry("CSkinnedMesh::[ERROR: UpdateSemantics() failed in ForceDeclRecursive() %d]", hr);
return;
} //end of if

pMeshContainer = (D3DXMESHCONTAINER_DERIVED *)pMeshContainer->pNextMeshContainer;
} //end of while loop

if(pFrame->pFrameSibling != NULL)
{
ForceDeclRecursive(pFrame->pFrameSibling);
} //end of if

if(pFrame->pFrameFirstChild != NULL)
{
ForceDeclRecursive(pFrame->pFrameFirstChild);
} //end of if
} //end of ForceDeclRecursive function








After you call ForceDeclRecursive() to make room for the tangents, then you can call ComputeTangents() to actually calculate them.

Another function that was somewhat helpful is this one which can be used for outputting the meshes vertex declaration...to make sure things look how they should after the conversion...just noticed that I should have used a switch() here...eh...*shrugs*

void OutputDecl(LPD3DXMESH pMesh)
{
HRESULT hr;
if(pMesh == NULL)
{
GameEngine.LogEntry("CSkinnedMesh::[ERROR in OutputDecl() pMesh == NULL");
return;
} //end of if

D3DVERTEXELEMENT9 elems[MAX_FVF_DECL_SIZE];
hr = pMesh->GetDeclaration(elems);
if(SUCCEEDED(hr))
{
GameEngine.LogEntry(" Outputting Vertex Declaration...");
GameEngine.LogEntry(" ----------------------------------------");
GameEngine.LogEntry(" NumBytesPerVert = %d", pMesh->GetNumBytesPerVertex());
for(int i = 0; i < MAX_FVF_DECL_SIZE; ++i)
{
// Did we reach D3DDECL_END() {0xFF,0,D3DDECLTYPE_UNUSED, 0,0,0}?
if(elems.Stream == 0xff)
break;

if(elems.Usage == D3DDECLUSAGE_POSITION)
{
GameEngine.LogEntry(" Element #%d - D3DDECLUSAGE_POSITION | Offset %d | Type %d | Usage Index %d", i, elems.Offset, elems.Type, elems.UsageIndex);
} //end of if
else if(elems.Usage == D3DDECLUSAGE_NORMAL)
{
GameEngine.LogEntry(" Element #%d - D3DDECLUSAGE_NORMAL | Offset %d | Type %d | Usage Index %d", i, elems.Offset, elems.Type, elems.UsageIndex);
} //end of if
else if(elems.Usage == D3DDECLUSAGE_TEXCOORD)
{
GameEngine.LogEntry(" Element #%d - D3DDECLUSAGE_TEXCOORD | Offset %d | Type %d | Usage Index %d", i, elems.Offset, elems.Type, elems.UsageIndex);
} //end of if
else if(elems.Usage == D3DDECLUSAGE_BLENDWEIGHT)
{
GameEngine.LogEntry(" Element #%d - D3DDECLUSAGE_BLENDWEIGHT | Offset %d | Type %d | Usage Index %d", i, elems.Offset, elems.Type, elems.UsageIndex);
} //end of if
else if(elems.Usage == D3DDECLUSAGE_TANGENT)
{
GameEngine.LogEntry(" Element #%d - D3DDECLUSAGE_TANGENT | Offset %d | Type %d | Usage Index %d", i, elems.Offset, elems.Type, elems.UsageIndex);
} //end of if
else if(elems.Usage == D3DDECLUSAGE_BLENDINDICES)
{
GameEngine.LogEntry(" Element #%d - D3DDECLUSAGE_BLENDINDICES | Offset %d | Type %d | Usage Index %d", i, elems.Offset, elems.Type, elems.UsageIndex);
} //end of if
else
{
GameEngine.LogEntry(" Element #%d contains unsupported usage...%d", i, elems.Usage);
} //end of else
} //end of for loop
} //end of if
else
{
GameEngine.LogEntry("CSkinnedMesh::[Failed to load vertex declaration]");
}//end of else
} //end of if








Like I said earlier, I just threw that out there with the hope of maybe saving somebody some time in the future.

Now for the results :-D

I purchased several "test models" since I don't have any normal maps for my in-game characters, I wanted to make sure I had everything working correctly before I spent time making the normal maps for my characters...I'll decribe the process I have to go through to create these normal maps later in the post.

I think it's working ;-)
This model is ~9k triangles with a 2k^2 color & a 2k^2 normal map.



I also got this police-officer-dude, he's ~4.5k triangles with a 1024x2048 normal & color map. [Left is with no normal mapping, right is with it enabled].


I have notice slight artifacts sometimes, I might have a issue in my shader, but I'm pretty sure the tangents are being generated correctly.

Ok like I said earlier, my meshes weren't created with normal maps, or even with normal mapping in mind. Thankfully Nvidia has that awesome tool which can generate a normal map from a texture. There are all kinds of tutorials on the net about how to use it.

Here is a image of the results of that tool....


I'm going to have to go through by hand touching up each of the converted normal maps...sounds like fun, hopefully I'll be able to get good results. The face is one of the areas that doesn't look right at all, straight outa the tool. Also I have to add more texture to the clothes in the normal map.

And the gangster in-game...not quite as impressive as the other 2 models, but it's an improvement [grin]


Another angle...


Well I'm tired of working on this now...but tommorow is another day, with more progress to be made. I'm devoting this week towards implementing the new order giving system into the game, and making a final push to complete the gameplay + interface.

- Dan

Ohhhh yea, here is 1 more screenshot...I think I have an idea for my next game lol ZOMG MONSTAR WARS.



Sign in to follow this  


4 Comments


Recommended Comments

Computing a basic tangent space is not hard, and Eric's code is fine for that. The challenge, however, is in properly smoothing the tangent space. Smoothing across mirrored and stretched tex coords is the tricky part.

You may look at NVMeshMender, which I co-wrote with Clint Brewer.

If your model is totally unwrapped, you won't have many seam or stretch issues, but if it is mirrored, you will see lighting issues unless you take that into account when you smooth the basis vectors across neigbhor tris.

Share this comment


Link to comment
Yes sir :-D

I forgot to mention that my step in between trying to use only D3DX functions to compute the tangents & calculating them myself was to use the meshmender lib in the Nvidia SDK.

I spent a day or two studying/playing with it, though I don't recall the show-stopper that made me give up move towards this option. Hmmmm.

Infact I was using the MeshMenderSimpleExample to preview the normal maps on my .x models when I was creating them...so it must have been something odd that caused me to give up.

Share this comment


Link to comment
I like normal mapping. ;-) The character does look tremendously better, especially the shirt. Good job on that, too bad you had so many problems with it.

Share this comment


Link to comment
Hurry up and finish Gang War so you can make that monster game ;D

I've been tracking this journal for a while, and you never fail to impress! Keep on rockin!

Share this comment


Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!