DX11 HLSL NaN headache

This topic is 468 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

Recommended Posts

After getting "black squares" rendered on my "bloom" post process:

So i searched and read this thread where i think L.Spiro had same problem.

Then i searched my HLSLs programs to find where i could have NaNs.

Found one problem in my point light shader:

float4 posVS = gbPosition_tex.Sample(sampPointClamp, tc);
// this has no effect !
if (IsNAN(posVS))
posVS = float4(0.0f, 0.0f, 0.0f, 1.0f);
if (any(isinf(posVS)))
posVS = float4(0.0f, 0.0f, 0.0f, 1.0f);

... //code skipped, nothing else touches posVS
float3 v = normalize(-posVS.xyz);

// but if i uncomment this there is NO more black squares !
//if (IsNAN(v))
//	v = float3(0.0f, 0.0f, 0.0f);
...// rest of lightning computation

IsNAN is defined like this (i cant use "native" isnan function because i cant set /Gis option inside VS2013 for fxc compiler options):

bool IsNAN(float n)
{
return (n < 0.0f || n > 0.0f || n == 0.0f) ? false : true;
}

bool IsNAN(float2 v)
{
return (IsNAN(v.x) || IsNAN(v.y)) ? true : false;
}

bool IsNAN(float3 v)
{
return (IsNAN(v.x) || IsNAN(v.y) || IsNAN(v.z)) ? true : false;
}

bool IsNAN(float4 v)
{
return (IsNAN(v.x) || IsNAN(v.y) || IsNAN(v.z) || IsNAN(v.w)) ? true : false;
}

wtf is going on?

Share on other sites

can you try:

bool IsNAN(float n)
{
return n != n;
}

Share on other sites

That gives me same results as my code.

Share on other sites

i just looked at the code a little closer:

posVS = float4(0.0f, 0.0f, 0.0f, 1.0f);
posVS == (0, 0, 0, 1)
float3 v = normalize(-posVS.xyz);


normalize:

a = abs(sqrt(v.x * v.x + v.y * v.y + v.z * v.z));
a == 0;

v.x = 0/0;
v.y = 0/0;
v.z = 0/0;

v.x == NaN;
v.y == NaN;
v.z == NaN;

IsNAN(v) == true

basically, if posVS has 0 in all three x, y, and z before you try to normalize it, then you'll end up with nans

Share on other sites
22 minutes ago, belfegor said:

That gives me same results as my code.

There is a isnan() intrinsic also ! Also, their is compiler flags that may relax rules over how to deal with math and change the result sometimes. No matter what, normalize of 0 is a bad idea

Share on other sites

@iedoc

We don't know how normalize is implemented in HLSL. But still i thought about that before and i tried even with this :

if (IsNAN(posVS))
posVS = float4(1.0f, 1.0f, 1.0f, 1.0f);
if (any(isinf(posVS)))
posVS = float4(1.0f, 1.0f, 1.0f, 1.0f);

rest of the code is the same, but same results.

cant use native isnan:

warning X3577: value cannot be NaN, isnan() may not be necessary.  /Gis may force isnan() to be performed

CANT set /Gis option in VS2013 fxc compiler options

Share on other sites

but what if gbPosition_tex.Sample(sampPointClamp, tc) returns (0, 0, 0, 1) or something (black)?

Share on other sites

Thats it.

float3 v = normalize(-(posVS.xyz + 0.0001f.xxx));

Thank you very much.

Share on other sites
1 minute ago, belfegor said:

Thats it.


float3 v = normalize(-(posVS.xyz + 0.0001f.xxx));

Thank you very much.

and what if posVS.xyz is -0.0001f ? You should probably do better with "float3 v = any(posVS!=0) ? normalize(posVS) : 0;" But how your position can be full zero in the first place ?

Share on other sites

"But how your position can be full zero in the first place ?"

Great question.Because i cleared that texture with 0's. I am thinking now what is better clear value to avoid future problems.

1. 1
2. 2
Rutin
20
3. 3
khawk
18
4. 4
A4L
14
5. 5

• 12
• 16
• 26
• 10
• 44
• Forum Statistics

• Total Topics
633763
• Total Posts
3013728
×