Thanks for the reply Jason. I did my best to implement your screen space method, and it almost works except I have one or two bugs that seem impossible. It's one of those times I really wish I could debug my shader. Here is my new hull shader:

PatchConstants PatchHS(InputPatch<VertexOutput,10> cp, uint patchID : SV_PrimitiveID) { PatchConstants pt; float4 corner0 = mul(float4(cp[0].PositionWorld,1.0f),ViewProjection); float4 corner1 = mul(float4(cp[1].PositionWorld,1.0f),ViewProjection); float4 corner2 = mul(float4(cp[2].PositionWorld,1.0f),ViewProjection); corner0 = corner0/(corner0.w + 0.00001f); corner1 = corner1/(corner1.w + 0.00001f); corner2 = corner2/(corner2.w + 0.00001f); const float dMax = 1.5f; float d0 = abs(distance(corner1.xy,corner2.xy)); pt.EdgeTess[0] = 64.0f * saturate(d0/dMax); float d1 = abs(distance(corner2.xy,corner0.xy)); pt.EdgeTess[1] = 64.0f * saturate(d1/dMax); float d2 = abs(distance(corner0.xy,corner1.xy)); pt.EdgeTess[2] = 64.0f * saturate(d2/dMax); float d3 = min(d0,min(d1,d2)); pt.InsideTess = 64.0f * saturate(d3/dMax); return pt; }

I have two problems that I can see in the following screen shot that are probably related somehow. The most obvious problem is that no matter how much space a triangle takes up on screen, my inside tess factor is computing to 2 or less even though I'm clearly trying to use the smallest edge length for the calculation. The second problem I am having is that I can see that the bottom and left sides of the triangles are matching up with their neighbor, but the right side is not! How can only one side not match up? If I switched them there should be two sides not matching up. It would seem that my d2 value is bad or NaN or something, but I can't imagine why that would be, also it seems to almost be right, while NaN values usually jump out at you.

[attachment=15215:TessProb.png]

I almost wonder if my code is being optimized away or something. Here is a picture of the resulting gaps. You can see they are only on the one diagonal.

[attachment=15217:TessProb2.jpg]