Implementing Parallax Occlusion Mapping

Now that we have a better understanding of the algorithm of parallax occlusion mapping, it is time to put our newly acquired knowledge to use. First we will look at the required texture data and how it is formatted. Then we will step through a sample implementation line by line with a thorough explanation of what is being accomplished with each section of code. The sample effect file is written in HLSL, but the implementation should apply to other shading languages as well.

Before writing the parallax occlusion map effect file, let's look at the texture data that we will be using. The only required data is that we need a height-map of the volumetric surface that we are trying to simulate. In this example, the height data will be stored in the alpha channel of the regular color texture map, with a value of 0 corresponding to the deepest point, and a value of 1 corresponding to the polygonal surface. Figure 4 shows the alpha channel height-map and the color texture that it will be coupled with.

Figure 4:  Texture data used

With the texture data understood, we will now look into the vertex shader to see how we set up the parallax occlusion mapping pixel shader.

The first step in the vertex shader is to calculate the vector from the eye (or camera) position to the vertex. This is done by transforming the vertex position to world space, and then subtracting its position from the eye position.

float4 VertexPositionWS = mul( float4(IN.position, 1), mW );
float3 P = VertexPositionWS.xyz;
float3 E = EyePositionWS.xyz - P;

Next, we must transform the eye vector and the vertex normal to tangent space. The transformation matrix that we will use is based on the vertex normal, binormal, and tangent vectors.

float3x3 tangentToWorldSpace;
tangentToWorldSpace[0] = mul( IN.tangent, mW );
tangentToWorldSpace[1] = mul( IN.binormal, mW );
tangentToWorldSpace[2] = mul( IN.normal, mW );

Each of these vectors is transformed to world space, and are then used to form the basis of the rotation matrix for converting from tangent to world space. Since this is a rotation only matrix, if we transpose the matrix it becomes its inverse. This produces the world to tangent space rotation matrix that we need.

float3x3 worldToTangentSpace = transpose(tangentToWorldSpace);

Now the output vertex position and the output texture coordinates are trivially calculated.

OUT.position = mul( float4(IN.position, 1), mWVP );
OUT.texcoord = IN.texcoord;

And finally, we use the world to tangent space rotation matrix to transform the eye vector and vertex normal to tangent space.

OUT.eye = mul( E, worldToTangentSpace );
OUT.normal = mul( IN.normal, worldToTangentSpace );

That is all there is for the vertex shader. Now we move on to the pixel shader, which contains the actual parallax occlusion mapping code. The first calculation in the pixel shader is to determine the maximum parallax offset length that can be allowed.  This is calculated in the same way that standard parallax mapping does. The maximum parallax offset is a function of the depth of the surface, as well as the orientation of the eye vector to the surface. For a further explanation see "Parallax Mapping with Offset Limiting: A Per-Pixel Approximation of Uneven Surfaces" by Terry Welsh.

float fParallaxLimit = length(IN.eye.xy) / IN.eye.z;
fParallaxLimit *= fHeightMapScale;

Next we calculate the direction of the offset vector. This is essentially a two dimensional vector that exists in the xy-plane. This must be the case, since the texture coordinates are on the polygon surface with z = 0 (in tangent space) for the entire surface. The calculation is done by finding the normalized vector in the direction of offset, which is essentially the vector formed from the x and y components of the eye vector. This direction is then scaled by the maximum parallax offset calculated in the last step.

float2 vOffset = normalize( -IN.eye.xy );
vOffset = vOffset * fParallaxLimit;

Now we must determine how many height-map samples we are going to take while determining where the eye vector intersects it. This is done by using a dot product of the surface normal and the eye vector as a measure of how 'straight on' the surface is to the viewing direction. First we find the normalized normal and eye vectors.

float3 E = normalize( IN.eye );
float3 N = normalize( IN.normal );

Then the number of samples is determined by lerping between a user specified minimum and maximum number of samples.

int nNumSamples = (int)lerp( nMinSamples, nMaxSamples, dot( E, N ) );

Since the total height of the simulated volume is 1.0, then starting from the top of the volume where the eye vector intersects the polygon surface the height is 1.0. As we take each additional sample, the height of the vector at the point that we are sampling is reduced by the reciprocal of the number of samples. This effectively splits up the 0.0-1.0 height into n chunks where n is the number of samples. This means that the larger the number of samples, the finer the height variation we can detect.

float fStepSize = 1.0 / (float)nNumSamples;

Since we would like to use dynamic branching in our sampling algorithm, we must not use any instructions that require gradient calculations within the dynamic loop section. This means that for our texture sampling we must use tex2Dgrad instead of a plain tex2D instruction. To use tex2Dgrad, we must manually calculate the texture coordinate gradients in screen space outside of the dynamic loop. This is done with the ddx and ddy instructions.

float2 dx, dy;
dx = ddx( IN.texcoord );
dy = ddy( IN.texcoord );

Now we initialize the required variables for our dynamic loop. The purpose of the loop is to find the intersection of the eye vector with the height-map as efficiently as possible. So when we find the intersection, we want to terminate the loop early and save the extra sampling efforts. We start with a comparison height of 1.0 (corresponding to the top of the volume), initial parallax offset vectors of (0,0), and starting at the 0^th sample.

float2 vOffsetStep = fStepSize * vOffset;
float2 vCurrOffset = float2( 0, 0 );
float2 vLastOffset = float2( 0, 0 );
float2 vFinalOffset = float2( 0, 0 );

float4 vCurrSample;
float4 vLastSample;

float stepHeight = 1.0;
int nCurrSample = 0;

Next comes the dynamic loop itself. For each iteration of the loop, we sample the texture coordinates along our parallax offset vector. For each of these samples, we compare the alpha component value to the current height of the eye vector. If the eye vector has a larger height value than the height-map, then we have not found the intersection yet. If the eye vector has a smaller height value than the height-map, then we have found the intersection and it exists somewhere between the current sample and the previous sample.

while ( nCurrSample < nNumSamples )
{
   vCurrSample = tex2Dgrad( Sampler, IN.texcoord + vCurrOffset, dx, dy );
   if ( vCurrSample.a > stepHeight )
   {
      float Ua = (vLastSample.a - (stepHeight+fStepSize))
                  / ( fStepSize + (vCurrSample.a - vLastSample.a));
      vFinalOffset = vLastOffset + Ua * vOffsetStep;

      vCurrSample = tex2Dgrad( Sampler, IN.texcoord + vFinalOffset, dx, dy );
      nCurrSample = nNumSamples + 1;
   }
   else
   {
      nCurrSample++;
      stepHeight -= fStepSize;
      vLastOffset = vCurrOffset;
      vCurrOffset += vOffsetStep;
      vLastSample = vCurrSample;
   }
}

Once the intersection samples have been found, we solve for the linearly approximated intersection point between the last two samples. This is done by finding the intersection of the two line segments formed between the last two samples and the last two eye vector heights. Then a final sample is taken at this interpolated final offset, which is considered the final intersection point.

Now all that is left is to illuminate the pixel based on these new offset texture coordinates. In our example here, we simply return the color that was sampled at this point. In the place of this diffuse color return value, you could use the offset texture coordinates to sample a normal map, gloss map or whatever to implement your favorite lighting model.

OUT.color = vSampledColor;

Now that we have seen parallax occlusion mapping at work, lets consider some of the parameters that are important to the visual quality and the speed of the algorithm.

Algorithm Metrics