Hello,

I'm doing tessellation and while I got the positions setup correctly using quads I am at a loss how to generate smooth normals from them. I suppose this should be done in the Domain shader but how would this process look like?

I am using a heightmap for tessellation but I would rather generate the normals from the geometry than use a normal map, if possible.

Cheers

18 hours ago, KaiserJohan said:

Hello,

I'm doing tessellation and while I got the positions setup correctly using quads I am at a loss how to generate smooth normals from them. I suppose this should be done in the Domain shader but how would this process look like?

I am using a heightmap for tessellation but I would rather generate the normals from the geometry than use a normal map, if possible.

Cheers

You should have access to "node" vertex normals, right (eg "pixel normals" on your heightmap, that you've precomputed)? If not, you might sample the heightmap in the vertex shader and calculate the normals in realtime.

Assuming quads, interpolating them in the tessellator should be as simple as bilerping the node normals. You can pass this information to the GPU as adjacency data and access it in the geometry shader*. Remember that since you're dealing with normals, the resulting vectors need to be renormalized after interpolation. 

* I don't have any experience with D3D, but searching for "vertex array adjacency" turns up this and this.

I managed to produce normals in the pixel shader using the screen-space derivates of the view position from the domain shader like this

[domain("quad")]
DomainOut domain_main(PatchTess patchTess, float2 uv : SV_DomainLocation, const OutputPatch<HullOut, 4> quad)
{
	DomainOut ret;

	float2 topMidpointWorld = lerp( quad[ 0 ].mWorldPosition.xz, quad[ 1 ].mWorldPosition.xz, uv.x );
	float2 bottomMidpointWorld = lerp( quad[ 3 ].mWorldPosition.xz, quad[ 2 ].mWorldPosition.xz, uv.x );
	float2 midPointWorld = lerp( topMidpointWorld, bottomMidpointWorld, uv.y );

	float3 topMidpointN = lerp( quad[ 0 ].mWorldNormal.xyz, quad[ 1 ].mWorldNormal.xyz, uv.x );
	float3 bottomMidpointN = lerp( quad[ 3 ].mWorldNormal.xyz, quad[ 2 ].mWorldNormal.xyz, uv.x );
	float3 midPointN = lerp( topMidpointN, bottomMidpointN, uv.y );

	float2 topMidpointTexcoord = lerp( quad[ 0 ].mTexcoord, quad[ 1 ].mTexcoord, uv.x );
	float2 bottomMidpointTexcoord = lerp( quad[ 3 ].mTexcoord, quad[ 2 ].mTexcoord, uv.x );
	float2 midPointTexcoord = lerp( topMidpointTexcoord, bottomMidpointTexcoord, uv.y );

	float y = quad[ 0 ].mWorldPosition.y;
	y += ( gHeightmap.SampleLevel( gPointSampler, midPointTexcoord, 0.0f ).r * gHeightModifier );

	ret.mPosition = float4( midPointWorld.x, y, midPointWorld.y, 1 );
	ret.mViewPosition = mul( gFrameView, ret.mPosition );
	ret.mPosition = mul( gFrameViewProj, ret.mPosition );

	ret.mTexcoord = midPointTexcoord;

	return ret;
}

struct PixelOut
{
	float4 mDiffuse : SV_Target0;
	float3 mNormal : SV_Target1;
};


PixelOut ps_main(DomainOut input)
{
	PixelOut ret;

	float3 dFdxPos = ddx( input.mViewPosition.xyz );
	float3 dFdyPos = ddy( input.mViewPosition.xyz );
	ret.mNormal = normalize( cross( dFdxPos, dFdyPos ) );
	ret.mNormal += 1.0f;
	ret.mNormal *= 0.5f;

	ret.mDiffuse = float4( 0.0f, 1.0f, 1.0f, 1.0f);

	return ret;
}

It gives me flat shading per tessellated vertex however

Is there a better way to produce actual smooth normals? This just isn't good enough ?

On 10/28/2018 at 1:15 PM, TeaTreeTim said:

You are changing some heights after generating normals. Trying to calculate a normal from original vertice normals plus changes made in tesselation will be complicated. I would either:

- Calculate in the pixel shader from heightmap sampling (slower performance better quality)

- Calculate in the domain shader based on heightmap sampling

- Calculate in pixel shader from normal map (a texture that has pregenerated normals as a colour value)

- Calculate in domain shader or geometry shader (if you use one) from normal map

 

Also you seem to have some flipped normals which is a hassle with tesselation. Since its a height map with no overhangs, you can just test like this:

if (normal.y < 0)

normal = -normal;

 

Hmm.. how am I changing the heights after generating the normals? It's all done in a previous shader stage?

> - Calculate in the domain shader based on heightmap sampling
I suppose this seems like a solid option. Will the normals be correctly interpolated in the pixel shader? Is there anything different from calculating normals in domain shader from say how you typically do it in a simple vertex shader?

Any pointers are much appreciated ?