Jump to content
  • Advertisement
Sign in to follow this  

[3ds Max] Directional Lightmap baking script problems

This topic is 1953 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi, i've been working on a script to bake directional lightmaps (or Radiosity Normal maps as Valve called it) using 3ds Max but i'm having issues with the lightmaps it produces.

The script modifies the normals of the objects to face the 3 basis vectors before baking the lightmaps, but I think there is something amiss when doing this as the lightmaps aren't producing the required result.

The 3 lightmaps for an unwrapped box end up coming out like this:


And the final lightmap with shader looks like this:


The code that modifies the normals is as follows:


-- computeTangentSpace function taken from HalfVector on cgsociety
-- The face position has been removed from the resulting matrices
-- http://forums.cgsociety.org/showpost.php?p=3881243&postcount=11
function computeTangentSpace obj =
    local theMesh = snapshotAsMesh obj
    local tSpace = #()
    -- Do we have to flip faces?
    local flip = false
    local indices = #(1, 2, 3)
    if dot (cross obj.transform.row1 obj.transform.row2) obj.transform.row3 <= 0 do (
        indices[2] = 3
        indices[3] = 2
        flip = true
    for nFace = 1 to theMesh.numFaces do (
        local face = getFace theMesh nFace
        local tface = getTVFace theMesh nFace
        local v1 = getVert theMesh face[indices[1]]
        local v2 = getVert theMesh face[indices[2]]
        local v3 = getVert theMesh face[indices[3]]
        local uv1 = getTVert theMesh tface[indices[1]]
        local uv2 = getTVert theMesh tface[indices[2]]
        local uv3 = getTVert theMesh tface[indices[3]]
        local dV1 = v1 - v2
        local dV2 = v1 - v3
        local dUV1 = uv1 - uv2
        local dUV2 = uv1 - uv3
        local area = dUV1.x * dUV2.y - dUV1.y * dUV2.x
        local sign = if area < 0 then -1 else 1
        local tangent = [0,0,1]
        tangent.x = dV1.x * dUV2.y - dUV1.y * dV2.x
        tangent.y = dV1.y * dUV2.y - dUV1.y * dV2.y
        tangent.z = dV1.z * dUV2.y - dUV1.y * dV2.z
        tangent = (normalize tangent) * sign
        local normal = normalize (getFaceNormal theMesh nFace)
        if flip do normal = -normal
        local binormal = (normalize (cross normal tangent)) * sign
        append tSpace (Matrix3 tangent binormal normal [0,0,0])
    delete theMesh
    return tSpace
function SetNormalsModifier nodeList &editNormalsMod lightmapType =
    normalBasis = [0, 0, 1]
    if (lightmapType == 1) then
        return 0
    else if (lightmapType == 2) then
        normalBasis = [sqrt (2.0 / 3.0), 0, 1.0 / (sqrt 3.0)]
    else if (lightmapType == 3) then
        normalBasis = [-(1.0 / (sqrt 6.0)), 1.0f / (sqrt 2.0), 1.0f / (sqrt 3.0)]
    else if (lightmapType == 4) then
        normalBasis = [-(1.0 / (sqrt 6.0)), -(1.0f / (sqrt 2.0)), 1.0f / (sqrt 3.0)]
    for j = 1 to nodeList.count do
        obj = nodeList[j]
        -- Get the face TBN's
        meshFaceTBNs = computeTangentSpace obj
        normalFaceCount = editNormalsMod.getNumFaces node:obj
        normalVertexCount = editNormalsMod.getNumNormals node:obj
        -- Create an array of arrays to contain the vertex normals for averaging later
        normalsListArray = #()
        for i = 1 to normalVertexCount do
            append normalsListArray #()
        for i = 1 to normalFaceCount do
            faceSelectionArray = #{i}
            normalSelection = #{}
            -- Get the normal indices of the selected face
            editNormalsMod.ConvertFaceSelection &faceSelectionArray &normalSelection node:obj
            -- Get the face TBN
            TBN = meshFaceTBNs        
            for k in normalSelection do
                -- Transform the normal basis into tangent space?
                normal = normalize(normalBasis * TBN)
                -- Append the new normal to a list for this vertex
                append normalsListArray[k] normal
        for i = 1 to normalVertexCount do
            normal = [0, 0, 0]
            -- Average the normals from the list for this vertex
            for k = 1 to normalsListArray.count do
                normal += normalsListArray[k]
            normal /= normalsListArray.count
            -- Set the vertex normal
            editNormalsMod.SetNormal i (normalize normal) node:obj


I've also attached the script as-is and a DirectX shader to test with. The script is set up as a macroscript so will need to be added as a button to your UI. The shader requires lightmap coords and normal texture coords in channels 1 and 2, and has a checkbox to state which is which.

I have a feeling there is a matrix transform i'm missing to get the normal bases from Direct3D's coordinate system to 3ds' but nothing i've tried has magically fixed the problem and I don't understand everything here well enough to know the exact reason why it doesn't work.

Apologies if this is the wrong forum, was toss up between graphics theory and tools :).

Share this post

Link to post
Share on other sites

I'm going to assume from the lack of replies that I've probably put this in the wrong sub-forum. If that is the case can a moderator move it to a more appropriate sub-forum please?

Share this post

Link to post
Share on other sites
Sign in to follow this  

  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!