It's Rocko!

Ah yes, haha, remember watching that in the mid-late 90's. Maybe worth a rewatch now that I'm old enough to actually understand it better than back then...

As for the displacement mapping I eventually settled on a rather simple approach for use with pre-existing triangle list meshes (it is by no means perfect but it seems that the "real" solution is really just having the 3D / texturing artist(s) being aware of your intents and have them author appropriately mapped bump maps and ensure corners actually are rounded, albeit with extremely short edges between corner vertices, as seemed to be the main point that the chapter in Zink, Pettineo & Hoxley (2011) as referred by unbird in the last post). Basically I average all disjoint vertex normals and create a 6-control patch index buffer that holds the initial triangle in the first 3 indices and then any "dominant" vertices sharing the position of these original vertices in the last 3 indices. The dominant vertices are arbitrarily chosen as the first found in the vertex data for a given position and are set to the same vertex as in the first three indices if there are no shared vertices at a given position. The dominant vertex is used to ensure that all overlapping vertices will sample the same value from the displacement map, while still allowing them to sample other textures by their UV coords.

Not perfect but good enough for some general-purpose examples.

As for actually employing this kind of displacement mapping in a more professional game / visual demo / what-have-you, the artist should ensure that there are *no* disjoint vertices (as said, such vertices can be moved slightly apart and be connected by a short edge, allowing the corners to appear mostly sharp) and a secondary set of texture coordinates should be provided for the displacement map / alternatively the displacement map should be handcrafted such that when displaced (with a reasonable strength / height factor), faces won't extrude through each others.

Of course there exists more elaborate ways as described in the previous posts, I just thought I would share this if anyone found it interesting, After tracking down a copy of that practical rendering book just for that tessellation chapter, it didn't say that much more about this particular problem, so might as well save others the trouble.

Hi,

i'm struggling with PN AEN now for two weeks and I'm not sure where my mistake could be... I'm pretty sure, the indices are correct, since I tried out unbirds exmaple and get the same indices. However my result looks like this:

I get the same result as with PN Triangles, no difference.

My TCS is:

#version 420
layout (vertices = 9) out;
in vec3 vp[];
in vec2 UV[];
in vec3 vn[];
out vec3 vpos[];
out vec2 outuv[];
out vec3 vnor[];
uniform vec3  tessLevelOuter;
uniform float tessLevelInner;

patch out Patch
{
    vec3 b210;
    vec3 b120;
    vec3 b021;
    vec3 b012;
    vec3 b102;
    vec3 b201;
    vec3 b111;
    vec3 n110;
    vec3 n011;
    vec3 n101;
	vec2 t110;
    vec2 t011;
    vec2 t101;
} OutPatch;

#define b300 vp[0]
#define b030 vp[1]
#define b003 vp[2]

#define n200 vn[0]
#define n020 vn[1]
#define n002 vn[2]

#define t200 UV[0]
#define t020 UV[1]
#define t002 UV[2]

void main()
{

	if( gl_InvocationID == 0 )
		{
			gl_TessLevelOuter[0] = tessLevelOuter[0];
			gl_TessLevelOuter[1] = tessLevelOuter[1];
			gl_TessLevelOuter[2] = tessLevelOuter[2];
			gl_TessLevelInner[0] = tessLevelInner;
		}
		vpos[gl_InvocationID]=vp[gl_InvocationID];
		vnor[gl_InvocationID]=vn[gl_InvocationID];
		outuv[gl_InvocationID]=UV[gl_InvocationID];
		
		if( gl_InvocationID == 0 )
    {

						 
						 
		OutPatch.b210 = (2.0 * b300 + b030 - dot( b030 - b300, n200 ) * n200 + 2.0 * vp[3] + vp[4] - dot( vp[4] - vp[3], vn[3] ) * vn[3] ) / 6.0;
        OutPatch.b120 = (2.0 * b030 + b300 - dot( b300 - b030, n020 ) * n020 +2.0 * vp[4] + vp[3] -dot( vp[3] - vp[4], vn[4] ) * vn[4] ) / 6.0;
        OutPatch.b021 = (2.0 * b030 + b003 - dot( b003 - b030, n020 ) * n020 +2.0 * vp[5] + vp[6] - dot( vp[6] - vp[5], vn[5] ) * vn[5] ) / 6.0;
        OutPatch.b012 = (2.0 * b003 + b030 - dot( b030 - b003, n002 ) * n002 +2.0 * vp[6] + vp[5] - dot( vp[5] - vp[6], vn[6] ) * vn[6] ) / 6.0;
        OutPatch.b102 = (2.0 * b003 + b300 - dot( b300 - b003, n002 ) * n002 +2.0 * vp[7] + vp[8] -dot( vp[8] - vp[7], vn[7] ) * vn[7] ) / 6.0;
        OutPatch.b201 = (2.0 * b300 + b003 - dot( b003 - b300, n200 ) * n200 + 2.0 * vp[8] + vp[7] -dot( vp[7] - vp[8], vn[8] ) * vn[8] ) / 6.0;
				 
		OutPatch.b111 = (OutPatch.b210 + OutPatch.b120 + OutPatch.b021 +
                         OutPatch.b012 + OutPatch.b102 + OutPatch.b201) / 4.0 - (b300 + b030 + b003) / 6.0;				 
						 
						 
						 

        const vec3 d0 = b030 - b300;
        const vec3 d1 = b003 - b030;
        const vec3 d2 = b300 - b003;
        const vec3 n0 = n020 + n200;
        const vec3 n1 = n002 + n020;
        const vec3 n2 = n200 + n002;
        const vec3 v0 = (2.0 * dot( d0, n0 ) / dot( d0, d0 )) * d0;
        const vec3 v1 = (2.0 * dot( d1, n1 ) / dot( d1, d1 )) * d1;
        const vec3 v2 = (2.0 * dot( d2, n2 ) / dot( d2, d2 )) * d2;
        OutPatch.n110 = normalize( n0 - v0 );
        OutPatch.n011 = normalize( n1 - v1 );
        OutPatch.n101 = normalize( n2 - v2 );
     }
}

Any ideas what I'm doing wrong?

Off the top of my head each individual face of your cube is tessellated and then displaced.

You need to ensure that the edge vertices are shared in each (subdivided) side-face or else these seams will occur since all vertices on the top face are displaced only along the up axis and all vertices of the front face are displaced only along the depth axis.

A simple solution is to displace along the vertex normals and ensure that whereever you have overlapping vertices (such as at the corners of a cube) you set the normal of all such vertices to the average of all "actual" vertex normals at that position. This will make the edges a bit more bulky but keep the faces connected.

My previous post in this thread (just above yours) describes how I solved this in a relatively simple way in more detail.

I'm not doing any Displayment mapping so far.

My TES looks like this:

	vec3 vp=vpos[0] * w * w * w +
                          b030 * u * u * u +
                          b003 * v * v * v + 
						  InPatch.b210 * 3.0 * w * w * u + 
                          InPatch.b120 * 3.0 * w * u * u +
                          InPatch.b201 * 3.0 * w * w * v +
                          InPatch.b021 * 3.0 * u * u * v +
                          InPatch.b102 * 3.0 * w * v * v +
                          InPatch.b012 * 3.0 * u * v * v +
                          InPatch.b111 * 6.0 * w * u * v; 

	vec3 vn= normalize( n200 * w * w + n020 * u * u + n002 * v * v +
                             InPatch.n110 * w * u + InPatch.n011 * u * v + InPatch.n101 * w * v );

About the screenshot above: I'm not applying any displacement yet. The cube simply cracks by smoothing the surface. As far as I understood this is a known issue with PN, but PNAEN shouldn't have this problem.

I know that you meant dominant UV to solve displacement map cracking, but I still have PN cracking with PNAEN.

I just fed a standard cube to my shader and it stays a cube with flat sides, no matter the tesselation factors (both PN and PNAEN). Makes sense if you use the axis-aligned normals and not averaged, like suggested several times (the bezier surfaces are indeed flat if all normals are equal). Maybe you stumbled across the same bug I did (vertex shader, not normalized normals therefore wrong control point calculation).

Edit: How does your PN behave for a nice model like a sphere ?

I just fed a standard cube to my shader and it stays a cube with flat sides, no matter the tesselation factors (both PN and PNAEN). Makes sense if you use the axis-aligned normals and not averaged, like suggested several times (the bezier surfaces are indeed flat if all normals are equal).

At first I used a standard cube generated by Maya and the cube stayed flat exactly as you said. Then I changed the normals of the cube like this:

PN got the cracks as expected, but PN doesn't look smooth like in the graphic above and looks 100% like PN.

Maybe you stumbled across the same bug I did (vertex shader, not normalized normals therefore wrong control point calculation).

I tried that, adding normalize in the Vertexshader, but no difference, still cracks.

Edit: How does your PN behave for a nice model like a sphere ?

Looks smooth, no cracks. No difference to flat tessealtion or PNAEN.

You might show your re-indexing code, maybe someone spots something. Also, my test case is probably too small.

Here is my indexing code:

void indexPNAEN(std::vector<unsigned short> indices, std::vector<glm::vec3> & in_vertices, std::vector<unsigned short> & out_indicesaen)
{
	out_indicesaen.resize(indices.size()*3); //step 1 Create an output IB that is 3 times the size of input IB.
	//step 2 c define edge
	struct Edge
	{
		glm::vec3 p[2];
		short ind[2];

		bool operator == (const Edge& o) const
		{
			if (ind[0] == o.ind[0] && ind[1] == o.ind[1])
				return true;

			if (p[0] == o.p[0] && p[1] == o.p[1])
			{

					return true;
			}

			return false;
		}

	/*	bool operator != (const Edge& o) const{
			return !(*this == o);
		}*/


		Edge reverse()
		{
			Edge returnedge;
			returnedge.p[0] = p[1];
			returnedge.p[1] = p[0];
			returnedge.ind[0] = ind[1];
			returnedge.ind[1] = ind[0];

			return returnedge;	
		}
	};

	struct KeyHasher
	{
		std::size_t operator()(const Edge& k) const
		{
			using boost::hash_value;
			using boost::hash_combine;

			// Start with a hash value of 0    .
			std::size_t seed = 0;

			// Modify 'seed' by XORing and bit-shifting in
			// one member of 'Key' after the other:
			for (int hashi = 0; hashi < 2; hashi++)
			{
				hash_combine(seed, hash_value(k.p[hashi].x));
				hash_combine(seed, hash_value(k.p[hashi].y));
				hash_combine(seed, hash_value(k.p[hashi].z));
				hash_combine(seed, hash_value(k.ind[hashi]));
			}

			// Return the result.
			return seed;
		}
	};



	std::unordered_map<Edge, Edge,KeyHasher> edges(indices.size()); 

	//step 2
	for (int i = 0; i < indices.size(); i += 3) //For each input Triangle in IB,
	{
		out_indicesaen[3 * i] = indices[i]; //i0
		out_indicesaen[3 * i + 1] = indices[i+1]; //i1
		out_indicesaen[3 * i + 2] = indices[i + 2]; //i2

		out_indicesaen[3 * i + 3] = indices[i]; //i0
		out_indicesaen[3 * i + 4] = indices[i + 1]; //i1
		out_indicesaen[3 * i + 5] = indices[i + 1]; //i1

		out_indicesaen[3 * i + 6] = indices[i+2]; //i2
		out_indicesaen[3 * i + 7] = indices[i + 2]; //i2
		out_indicesaen[3 * i + 8] = indices[i]; //i0

		//2b and 2d

		Edge edge0;
		edge0.p[0] = in_vertices[indices[i + 1]];
		edge0.p[1] = in_vertices[indices[i]];
		edge0.ind[1] = indices[indices[i]];
		edge0.ind[0] = indices[indices[i + 1]];
		edges.emplace(edge0.reverse(), edge0);

		Edge edge1;
		edge1.p[0] = in_vertices[indices[i + 2]];
		edge1.p[1] = in_vertices[indices[i + 1]];
		edge1.ind[1] = indices[indices[i + 1]];
		edge1.ind[0] = indices[indices[i + 2]];
		edges.emplace(edge1.reverse(), edge1);

		Edge edge2;
		edge2.p[0] = in_vertices[indices[i]];
		edge2.p[1] = in_vertices[indices[i + 2]];
		edge2.ind[1] = indices[indices[i + 2]];
		edge2.ind[0] = indices[indices[i]];
		edges.emplace(edge2.reverse(), edge2);
	}


	//step 3 Walk the output index buffer (OB) constructed in step 2. For each patch of 9 indices:
	for (int i = 3; i < out_indicesaen.size(); i += 9)
	{

		//3a For each Edge in the current Patch, perform a lookup into Edge->Edge mapping created in step 2d.
		for (int k = 0; k < 6; k += 2)
		{
			int i0 = out_indicesaen[i + k];
			int i1 = out_indicesaen[i + k + 1];
			Edge temp;
			temp.ind[0] = i1;
			temp.ind[1] = i0;
			temp.p[0] = in_vertices[i1];
			temp.p[1] = in_vertices[i0];

			auto foundIt = edges.find(temp);
			if (foundIt!=edges.end()) //look up in edge vector
			{
				const Edge& second = foundIt->second;
				out_indicesaen[i + k] = second.ind[1];
				out_indicesaen[i + k + 1] = second.ind[0];
				
			}

		}

	}

}

Can you put that cube up for download (preferably in a common exchange format like Wavefront) ? I'll run it through my reindexer and give you the results.

Oh, thank you! IThe cube can be downloaded here. (I'm not permitted to attach an .obj file ). I'm starting to get frustrated with PNAEN...I can't find the bug for two weeks now. It's part of my master thesis.

Ok. Had to adjust the import (Assimp, join identical vertices), so only 24 instead of 36 distinct vertices. In any case, I included a dump of the vertices, so you could reconstruct the mesh programmatically. It's a bloated html-log (and the indices are dumped as tables and "raw" for easier copying).

From a quick glance, I think your reindexer is fine.

Edit: Wait, try something first. Hash only the positions (x,y,z), the map will fail otherwise (at least it did for my C# dictionary).

My reindexer is indeed wrong, I get totally different indices. I copied your indices manually into the vector and the cracks are gone! The shader works!

I'll try to hash only the positions. I hope that's the bug! Thank you very much for your help, the html log is awesome!

Edit: I found out that my edge Map seems to be already wrong. i have only 32 entried while your map has 36 entries.