Normal encoding problems

Started by
9 comments, last by kauna 11 years, 7 months ago
I've been trying to correctly encode my view-space normals for a while now but it's just not working.
I tried every algorithm in here (http://aras-p.info/t...rage.html#intro) but every time I get wrong normals.
Well except the basic one.

half4 encode (half3 n, float3 view)
{
return half4(n.xyz*0.5+0.5,0);
}
half3 decode (half4 enc, float3 view)
{
return enc.xyz*2-1;
}


This works perfectly but my specular keeps jumping around. I guess that would be a precision issue because of the 8bit target.
Well so then I try the next easiest thing which is storing X&Y and reconstructing Z like this:

half4 encode (half3 n, float3 view)
{
return half4(n.xy*0.5+0.5,0,0);
}
half3 decode (half2 enc, float3 view)
{
half3 n;
n.xy = enc*2-1;
n.z = sqrt(1-dot(n.xy, n.xy));
return n;
}


And it's error all over again. The lighting changes with my view direction. (where I look at)
I tried this with both an R8G8B8A8 and R16G16 format but no change.
How is it that every encoding method I try fails.
Any idea what I'm fundamentally doing wrong ?

Normals are calculated this way:

float4x4 WorldView = mul(World, View);

// Output view space normals
output.Normal.xyz = mul(input.Normal, (float3x3)WorldView);
Advertisement
You should ensure to input only normalized normals to your encoding function, if you take the interpolated vertex normals you might encounter artifacts. Test if this works:

half4 encode (half3 n, float3 view)
{
half3 tmp = nomalize(n);
return half4(tmp.xy*0.5+0.5,0,0);
}
Nope, still not correct somehow : /

Also with this algorithm the z direction seems to be inverted.

Added video showcasing the problem:
[media]
[/media]
(Diffuse only)
Directional light source is shining towards (0, 0, -1) so only the front face of the cube should be lit.

And here's the pixel shader code:

float4 encodedNormals = NormalTarget.Sample(PointSampler, input.UV);
float3 N = decode(encodedNormals);
float3 LightDir = mul(_LightDirection, View);
float NdotL = saturate(dot(N, -LightDir));
lighting.rgb = float3(NdotL, NdotL, NdotL);
You can't use normal.z=sqrt(1-....) because you will lose the sign of the z component (even in view space there can still be normals that have z component negative).
Have you tried debugging with PIX?

Or at the very least drawing your encoded normal rendertarget so you can look at it and see if the problem lies in the encoding or the decoding.
http://aras-p.info/texts/CompactNormalStorage.html

try euler angles and/or spheremap :)
It may be that you'll need to negate your normals for the compression algorithm and negate them again in the decompression algorithm.

Also, ensure that your normals are normalized.

Cheers!
Okay so I've tried them all again (except per pixel view space [method #8]) and the only thing that seems to work without any problems is the spherical coordinates one.
stereographics projection doesn't work (view dependent and wrong direction), same with the spheremap (mittring). The lambert-azimuthal equal area projection works almost but has some error with specific viewing angles (the NdotL becomes 0 suddenly for a second).

So I guess I'll be using the spherical coordinates for now but I still don't understand how more than half of the algorithms on this page just doesn't work for me.
If I could I'd probably use the spheremap transform...does anyone have an idea why this doesn't work ?
And yes sure I have been debugging this for a while with PIX I'm not seeing anything exceptional there that you couldn't already tell when going through the encoding/decoding in your head/on paper.

Also a little off topic but should bloom and tonemapping be used on specular ? It does seem to loose quite a bit of "energy" and form when going through those passes.
I use RG16_FLOAT target for the normals and the following code adapted from the stereographic projection. I had to negate the normals to make it work correctly.


#define STEREOSCALE 1.77777f
#define INVSTEREOSCALE (1.0f/STEREOSCALE)
float3 StoreNormal(float3 n)
{
n = normalize(-n);
float2 enc = (n.xy / (n.z+1)) * INVSTEREOSCALE;

return float3(enc,0);
}
float3 RestoreNormal(float3 enc)
{
float scale = STEREOSCALE;

float3 nn = enc.xyz * float3(scale,scale,0);
nn.z = 1.0f;
float g = 2.0 / dot(nn.xyz,nn.xyz);
float3 n;
n.xy = g*nn.xy;
n.z = g-1;
return normalize(-n);
}


Cheers!
@kauna that actually works tongue.png
I'm not sure why as I normalize before passing the normals to the encode function and after I decode them but that normalize and negation seems to be the key. Just negating alone doesn't do the trick. At least for the stereographic projection. Tried the same on the spheremap but didn't work as good unfortunately ;p

Thanks a lot though smile.png

btw what is it that you're doing different with the algorithm than the version on aras page ?
I see you calculate the stereoscale yourself but there appears to be a few other changes.

Also is it correct to sample the encoded normals bilinear ?

This topic is closed to new replies.

Advertisement