HLSL NaN headache

Started by
9 comments, last by belfegor 6 years, 7 months ago

After getting "black squares" rendered on my "bloom" post process:

3Q68R1n.png

 

So i searched and read this thread where i think L.Spiro had same problem.

 

Then i searched my HLSLs programs to find where i could have NaNs.

Found one problem in my point light shader:


float4 posVS = gbPosition_tex.Sample(sampPointClamp, tc);
    // this has no effect !
	if (IsNAN(posVS))
		posVS = float4(0.0f, 0.0f, 0.0f, 1.0f);
	if (any(isinf(posVS)))
		posVS = float4(0.0f, 0.0f, 0.0f, 1.0f);

    ... //code skipped, nothing else touches posVS
	float3 v = normalize(-posVS.xyz);

     // but if i uncomment this there is NO more black squares ! 
	//if (IsNAN(v))
	//	v = float3(0.0f, 0.0f, 0.0f);
    ...// rest of lightning computation

Please read comments in code.

IsNAN is defined like this (i cant use "native" isnan function because i cant set /Gis option inside VS2013 for fxc compiler options):


bool IsNAN(float n)
{
    return (n < 0.0f || n > 0.0f || n == 0.0f) ? false : true;
}

bool IsNAN(float2 v)
{
    return (IsNAN(v.x) || IsNAN(v.y)) ? true : false;
}

bool IsNAN(float3 v)
{
    return (IsNAN(v.x) || IsNAN(v.y) || IsNAN(v.z)) ? true : false;
}

bool IsNAN(float4 v)
{
    return (IsNAN(v.x) || IsNAN(v.y) || IsNAN(v.z) || IsNAN(v.w)) ? true : false;
}

wtf is going on? 

Advertisement

can you try:


bool IsNAN(float n)
{
    return n != n;
}

 

That gives me same results as my code.

i just looked at the code a little closer:


posVS = float4(0.0f, 0.0f, 0.0f, 1.0f);
posVS == (0, 0, 0, 1) 
float3 v = normalize(-posVS.xyz);

normalize:


a = abs(sqrt(v.x * v.x + v.y * v.y + v.z * v.z));
a == 0;

v.x = 0/0;
v.y = 0/0;
v.z = 0/0;

v.x == NaN;
v.y == NaN;
v.z == NaN;

IsNAN(v) == true

basically, if posVS has 0 in all three x, y, and z before you try to normalize it, then you'll end up with nans

 

 

 

 

22 minutes ago, belfegor said:

That gives me same results as my code.

There is a isnan() intrinsic also ! Also, their is compiler flags that may relax rules over how to deal with math and change the result sometimes. No matter what, normalize of 0 is a bad idea :)

@iedoc

We don't know how normalize is implemented in HLSL. But still i thought about that before and i tried even with this :


if (IsNAN(posVS))
		posVS = float4(1.0f, 1.0f, 1.0f, 1.0f);
	if (any(isinf(posVS)))
		posVS = float4(1.0f, 1.0f, 1.0f, 1.0f);

rest of the code is the same, but same results.

 

@galop1n 

cant use native isnan:


warning X3577: value cannot be NaN, isnan() may not be necessary.  /Gis may force isnan() to be performed

CANT set /Gis option in VS2013 fxc compiler options

but what if gbPosition_tex.Sample(sampPointClamp, tc) returns (0, 0, 0, 1) or something (black)?

Thats it. 


float3 v = normalize(-(posVS.xyz + 0.0001f.xxx));

Thank you very much.

1 minute ago, belfegor said:

Thats it. 



float3 v = normalize(-(posVS.xyz + 0.0001f.xxx));

Thank you very much.

and what if posVS.xyz is -0.0001f ? You should probably do better with "float3 v = any(posVS!=0) ? normalize(posVS) : 0;" But how your position can be full zero in the first place ?

"But how your position can be full zero in the first place ?"

Great question.Because i cleared that texture with 0's. I am thinking now what is better clear value to avoid future problems.

This topic is closed to new replies.

Advertisement