Jump to content
  • Advertisement
Sign in to follow this  

Phong tessellation not smooth?

This topic is 624 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts



I'm implementing Phong tessellation to smooth out my terrain a bit. I've tried two completely different implementations, and I'm pretty sure I'm doing everything correct. I've implemented the phong tessellation technique from http://onrendering.blogspot.se/2011/12/tessellation-on-gpu-curved-pn-triangles.html, essentially a copy-paste job, and it's identical to the "homemade" solution I was using before. I had a problem where I wasn't normalizing the normal in the vertex shader (it got scaled by the normal matrix), which messed up the phong calculations, but I've now ensured that that doesn't happen.


I found the problem when I noticed that my SSAO was still giving triangle shaped artifacts on smooth curved surfaces. As a test, I tried rendering the triangle normal calculated per pixel using normalize(cross(dFdx(vViewPosition), dFdy(vViewPosition))), and this lead me to the following results.











As you can see, phong tessellation seems to add a certain level of smoothness inside each triangle, but the triangle edges have completely different normals. I don't get how this could happen. Like I said, I'm 99.99% sure my tessellation implementation is correct since it's pretty much copy-pasted from the article I linked above AND my other implementation looks identical.


Tessellation control:

out float termIJ[];
out float termJK[];
out float termIK[];

float phong(int i, vec3 q){
	vec3 q_minus_p = q - vViewPosition[i];
	return q[gl_InvocationID] - dot(q_minus_p, vNormal[i])  * vNormal[i][gl_InvocationID];

//In main()

	termIJ[gl_InvocationID] = phong(0, vViewPosition[1]) + phong(1, vViewPosition[0]);
	termJK[gl_InvocationID] = phong(1, vViewPosition[2]) + phong(2, vViewPosition[1]);
	termIK[gl_InvocationID] = phong(2, vViewPosition[0]) + phong(0, vViewPosition[2]);

Tessellation evaluation:

in float termIJ[];
in float termJK[];
in float termIK[];

#define tc1 gl_TessCoord

//in main()

	vec3 tc2 = tc1*tc1;
	vec3 tIJ = vec3(termIJ[0], termIJ[1], termIJ[2]);
	vec3 tJK = vec3(termJK[0], termJK[1], termJK[2]);
	vec3 tIK = vec3(termIK[0], termIK[1], termIK[2]);
	vViewPosition = tc2[0]*tViewPosition[0] + tc2[1]*tViewPosition[1] + tc2[2]*tViewPosition[2] + 
			  + tc1[0]*tc1[1]*tIJ + tc1[1]*tc1[2]*tJK + tc1[2]*tc1[0]*tIK;

Hence, my question is: is this a limitation of phong tessellation or is there something wrong with my vertex inputs (or even my implementation)?

Share this post

Link to post
Share on other sites
It is indeed a downside of Phong tessellation: It does not guarantee C1 continuity across patches.
You can tweak it a bit by lerping it with the untesselated patch (s. here, alpha parameter e.g. = 3/4).

Share this post

Link to post
Share on other sites

Thanks a lot for the info!


I tried using an alpha of 3/4 and it did improve it a tiny bit, but I'm just not sure if it's good enough yet. Is there some other technique which has higher quality and produces a continuous result? What about those PN triangles in the article about phong tessellation? Are there any other good techniques for smoothing the triangles? Due to the way the texturing works, I can't smooth it with a height map.

Share this post

Link to post
Share on other sites

PN Triangles should do the trick, but it's of course more costly.  :wink:


Edit: Kudos for the debugging approach.

Edited by unbird

Share this post

Link to post
Share on other sites

I implement PN triangle tessellation; made a better implementation than the article I linked using the math it listed.


Here's the surface textured but with triangle face normals. Ignore the texture stretching.



Here is the same thing drawn with PN tessellation, still with face normals on (still the normals of the actual geometry):




I'm a bit worried though; I needed to use an alpha value of 1/3 to get this result, or all the triangles just looked super puffy. It also doesn't handle all edges continuously, as you can see on the bulge to the right. Is this a limitation of PN triangles or do I have a bug somewhere?


Here are my shaders. Sorry, they're not very readable...


Tessellation control:

out vec3 tPN3[];
out vec3 tPN2[];
out vec3 tPN1[];

//in main()

	int i0 = gl_InvocationID;
	int i1 = (gl_InvocationID + 1) % 3;
	vec3 p0 = vViewPosition[i0];
	vec3 p1 = vViewPosition[i1];
	vec3 n0 = vNormal[i0];
	vec3 n1 = vNormal[i1];
	vec3 dp = p1 - p0;
	tPN3[gl_InvocationID] = p0;
	tPN2[gl_InvocationID] = p0*(2.0/3.0) + p1*(1.0/3.0) - dot(dp, n0) * n0;
	tPN1[gl_InvocationID] = p0*(1.0/3.0) + p1*(2.0/3.0) - dot(-dp, n1) * n1;

Tessellation evaluation:

in vec3 tPN3[];
in vec3 tPN2[];
in vec3 tPN1[];

//in main()

	vec3 b300 = tPN3[0];
	vec3 b030 = tPN3[1];
	vec3 b003 = tPN3[2];
	vec3 b210 = tPN2[0];
	vec3 b021 = tPN2[1];
	vec3 b102 = tPN2[2];
	vec3 b120 = tPN1[0];
	vec3 b012 = tPN1[1];
	vec3 b201 = tPN1[2];
	vec3 e = (b210 + b021 + b102 + b120 + b012 + b201) * (1.0/6.0);
	vec3 v = (b300 + b030 + b003) * (1.0/3.0);
	vec3 b111 =  e + 0.5*(e-v);
	vec3 rawPosition = interpolate(tPN3);
	tc2 *= 3.0;
	vec3 pnPosition = 
				tc3[0] * b300 +
				tc3[1] * b030 +
				tc3[2] * b003 +
				tc2[0] * tc1[1] * b210 +
				tc2[1] * tc1[2] * b021 +
				tc2[2] * tc1[0] * b102 +
				tc1[0] * tc2[1] * b120 +
				tc1[1] * tc2[2] * b012 +
				tc1[2] * tc2[0] * b201 +
				6.0 * tc1[0] * tc1[1] * tc1[2] * b111;
	vViewPosition = mix(rawPosition, pnPosition, alpha); //alpha=1.0/3.0

Thanks a lot for everything, unbird! You've been tremendously helpful!



Edit: What do you mean by "debugging approach"?

Edited by theagentd

Share this post

Link to post
Share on other sites

What do you mean by "debugging approach"?

Using face normals to visualize the continuity. Hint: Output normals as colors directly, this gives a even clearer picture, e.g. color.rgb = normal * 0.5 + 0.5;

As for your current solution: Careful now. I don't know what you have there, but it can't be a proper PN Triangle tessellation, it looks way too simple.

Also, using the lerp trick is rather counterproductive: PN triangle doesn't need it. You immediately break the continuity now. The point of PN Triangle is that you can throw any mesh (with proper matching normals) at it and it just works. With the alpha-lerp you now have veiled a bug.

Go try using that article's PN Triangle code as is.

Share this post

Link to post
Share on other sites

I found the bug. The calculation for tPN2 and tPN1 wasn't multiplying the dot-product result by 1/3, which is why I needed that alpha of 1/3 to compensate. I no longer need a 1/3 alpha, but there are still discontinuities in the normal.


The continuity is correct at the 3 corners, but between the corners the normals gets more and more different between triangles. For a somewhat tessellated sphere it pretty much 100% smooth, but it doesn't turn a cube with smoothed normals into a shape with smooth normals.



(I tried outputting the normal as a color as you suggested, but it was difficult for me to see the color difference. In the pictures below, I use a simple dot(normal, viewPosition) fake diffuse light to visualize the normal.)











However, compared to this screenshot I dug up from the Unreal Engine docs, it looks pretty much the same.





I have checked and rechecked the math a million times, and it's all correct. The reason why my code is shorter is:


1. because I don't parallelize over the XYZ-coordinates, instead parallelizing over control points. The article I linked computes one of X, Y or Z of all 10 control points in each invocation of the shader, while mine computes one corner and the two control points between the i-th and the (i+1)%3-th control point per invocation, while the last 10 control point is computed in the evaluation shader (for now). This turns the article's 6 dot products per invocation to just 2 per invocation and is much less code.


2. because I don't do the quadratic normal interpolation for the vertices yet, as I'm testing the smoothness of the generated vertex positions.



From what I can tell, I seem to have hit the limit of current tessellation algorithms. =/ Well, it's better than Phong tessellation at least, but still not perfect. Thank you so much for your help, unbird!

Share this post

Link to post
Share on other sites
Wow. Congrats. I stand corrected. Looks like a very slim solution, I'm impressed. At least I sensed a bug there. :P

I think you're fine. That's about as good as it gets. Those edges look suspicious but also consider this: It's PN Triangles. Those cubic patches might fit your mesh - or not. E.g. teh teapot can look quite bad (it should be quad patches, it's defined that way). And there's always other schemes (NURBS, Catmull-Clarke) if that's warranted.

I'm curious now, if I may ask: Has the AO improved ? And what's that mesh: Marching cubes?

PS: Sorry about the visualization hint, it occasionally helps. I should also admit I don't really do OpenGL, so bear with me. :D

Share this post

Link to post
Share on other sites

Thank you so much for all your help!


Here's the view of the SSAO before and after tessellation. Note that I still haven't implemented the quadratic normal interpolation, so this isn't the final look of the SSAO with tessellation.


Tessellation off:




Tessellation on:



It's not perfect, but a lot of the remaining problems are due to the original mesh. I'm quite happy with the result; hopefully the normal interpolation will make it even better


I'm afraid I can't really discuss the mesh. =/ It's a "new" way of making terrain meshes (as in probably 10 000 other people have coded the same and I just don't know about it), but we'll see what comes of it.

Share this post

Link to post
Share on other sites
You're welcome :D

Thanks for the pix. The quadratic normals will be the last piece of the puzzle. But yeah, from there I think only a higher resolution base mesh or another AO approach can help.

Good luck with your terrain. Looks promising.

Share this post

Link to post
Share on other sites
Sign in to follow this  

  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!