# Displacement Mapping Gone Wrong (solved)

This topic is 1882 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

Ok, so I'm taking my first stab at displacement mapping, which involves sampling height maps in the domain shader. I'm trying to use textures to carve river valleys into a hex game board. My first attempt at trying to find a per patch edge mip level selection was totally worthless. For now, I am using the following code to simply sample mip level 0,

	if(dout.RiverTexCoord.z > 0.0)
{
//float mipLevel = clamp(pc.MipLev[0] * w + pc.MipLev[1] * u + pc.MipLev[2] * v, 0.0f, 10.0f);
float4 rivColor = riverTextures.SampleLevel(samAnisotropic,dout.RiverTexCoord,0.0f);
position.y -= rivColor.r * 0.3;
}



I don't want the displacement to happen against the interpolated normal, in this case I want it to move straight down some small amount. The problem with sampling things this way and my previous ways is that when the camera moves, the vertices bubble and boil in a very disturbing way. Here is a video of what is happening. Watch it in HD so that the artifacts aren't smoothed away.

Clearly, displacement mapping has to be done exactly right in terms of selecting the proper mip level. Even with only sampling one mip level, I'm getting all kinds of cracks and general instability. How can I do this correctly so that it looks good?

##### Share on other sites

Ok, I have greatly improved (not eliminated) the swimming artifacts by selecting the proper mip level. Choosing the mip level is closely related to how you chose your tessellation factors. These are triangle patches, so I choose tessellation factors for 3 edges. I take the screen length of each edge and tessellate from 1 to 64 based on that length for each edge, and then for the inside tess factor I choose the least of those. Here is how I am using that to calculate mip levels.  In my hull shader I store these values in my patch constants struct:

	pt.MipLev[0] = log2(texWidth/pt.EdgeTess[0]);
pt.MipLev[1] = log2(texWidth/pt.EdgeTess[1]);
pt.MipLev[2] = log2(texWidth/pt.EdgeTess[2]);
pt.MipLev[3] = log2(texWidth/pt.InsideTess);



Since the texel length of one side of my triangle patch is always 512, even if it's foreshortened or whatever, I can divide this by the tess factor and get what my mip level is supposed to be. This is the log2(L/T) function from this article

The next step is to make sure that every patch edge is using the exact same mip level as the one touching it and also that it is the appropriate mip level for the tessellation that is going on. Here is how I am doing this in the domain shader:

	float A = V/(U + V + 0.00001f);
float B = W/(V + W + 0.00001f);
float C = U/(U + W + 0.00001f);

if(A > 0.99f)
{
mipLevel = pc.MipLev[0];
}
else if(B > 0.99f)
{
mipLevel = pc.MipLev[1];
}
else if(C > 0.99f)
{
mipLevel = pc.MipLev[2];
}
else
{
mipLevel = pc.MipLev[3];
}
float4 rivColor = riverTextures.SampleLevel(samAnisotropic,dout.RiverTexCoord,mipLevel);
position.y -= rivColor.r * 0.3;



The terms A, B and C refer to the closeness of a patch edge, and is related to the barycentric coordinates of the triangle. When I am on a patch edge I use the mip level from that edge, which is the same for the adjacent patch. If I'm not close to an edge I use the mipLev from the inside tess factor. To insure that my mip levels were matching up edge to edge, I used vertex coloration as a debugging tool to verify this like so.

	if(mipLevel > 9.0f)
dout.VertexColor = float4(1,0,0,1);
else if(mipLevel > 8.0f)
dout.VertexColor = float4(0,1,0,1);
else if(mipLevel > 7.0f)
dout.VertexColor = float4(0,0,1,1);
else if(mipLevel > 6.0f)
dout.VertexColor = float4(1,1,0,1);
else if(mipLevel > 5.0f)
dout.VertexColor = float4(0,1,1,1);
else if(mipLevel > 4.0f)
dout.VertexColor = float4(1,0,1,1);
else if(mipLevel > 3.0f)
dout.VertexColor = float4(1,0,0,1);
else if(mipLevel > 2.0f)
dout.VertexColor = float4(0,1,0,1);



This color scheme showed that my edges were matching up across patches.

While I have greatly diminished the swimming artifacts by choosing better mip levels, I still have a whole bunch of unacceptable cracks!

I think I know why. Even though my mip levels are matching up, each patch is using a different texture that, though they are designed to fit together and do so just fine from a color perspective, might not exactly fit together enough for this use. My heightmap is just a grayscale png with all values matching from 0 to 255. That's not very much vertical resolution. If one edge reads a 255 while its neighbor reads 254, that's a crack! Right?

Unfortunately, I am using GDI+ and C# to make my current heightmaps programatically, and I'm sure there's no support for different floating point texture formats. I'll have to learn how to make float32 textures, which is probably not that easy, and all of that work just to test my theory! Blegh. If anyone thinks my theory is unsound, please feel free to save me from myself and stop this madness by speaking up!

EDIT: Ok, never mind. I think I have a different problem. If I were using square patches, I could easily ensure that my textures lined up exactly end to end with the same data. However, since these are triangle patches, I am lining them up with 60 degree rotations of the texture maps. There's no way to do that and ensure that one patch edge will get the same exact sample as it's neighbor. Yuck.

Edited by cephalo

##### Share on other sites
And again you found the culprit on your own. Congrats

Now I wonder if you couldn't "just" tile/splat differently, without rotation. Consider this. Or rather think about tiling the triangles directly. Likely needs some playing with texcoords (different scale for the axis or something), but I think it's possible.

Edit: Maybe I'm not thinking straight. Could still mean you need to blend at the edges. Lots of texture reads ?

GDI+ ? It has 16 bit format support (per channel), never used it, but maybe it helps. Still not float though, if you really need to. Could you do it with D3D ?

##### Share on other sites

I think I have the issue nailed down. I just have to fix it in a way that is not overly complicated in the shader and not too memory hogging overall. I want to keep my instance data small so I can build huge, huge maps.

Here is the issue with displacement mapping when your patch is designed to take a whole texture. Generally when we design tiling textures, we design them so that the edges are adjacent to the other edge. For example, the left edge is designed to flow seamlessly into the right edge for a tiling texture. Surface patches however, lie on top of each other at the edge, NOT adjacent to one another! This problem would exist even with square patches in my case.

No matter what kind of texture you use, even a high precision one, it is vital that shared patch edges sample the same exact data. Close is not good enough in this case, as the hardware seems to like to accentuate these cracks and even from far away they look like blinking Christmas tree lights.

The fact that I'm using square textures on a triangle patch complicates things, but there is one thing that is in my favor. All of the complex river interactions happen on the interior of the textures. At the patch edges, the only thing that ever happens is that one of three river sizes or none will cross there, and also they might be flipped. So I could sample either one of four 1D textures there or even use certain of my simpler textures at uv = (u, 0). I just have to devise a scheme for the shader to know what river size is needed and whether its coordinate should be inverted.

EDIT: I should add that these issues wouldn't exist if your texture border weren't co-mingling with your patch border. I could layer a rocky texture over the whole terrain without cracking. This only comes up if you are trying to build large textures out of pieces made to fit together at patch borders.

[attachment=18891:TilingPatches.jpg]

Edited by cephalo

##### Share on other sites

Ok, I've got this issue solved. I have the river sizes for each edge as part of the input for the vertex shader. Then in the hull shader patch constant function I determine the texture to choose and also if the texture should be flipped, and then in the domain shader I interpolate the coordinates for sampling the (u,0.0f) row of the texture which represents the river crossing.

Another issue is of course getting the right mip level at the patch corners, I cheated on this and simply used a zero value which is correct for this particular scheme only.

• ### What is your GameDev Story?

In 2019 we are celebrating 20 years of GameDev.net! Share your GameDev Story with us.

• 28
• 16
• 10
• 10
• 11
• ### Forum Statistics

• Total Topics
634107
• Total Posts
3015549
×