Jump to content
  • Advertisement
Sign in to follow this  
belfegor

DX11 HLSL NaN headache

Recommended Posts

After getting "black squares" rendered on my "bloom" post process:

3Q68R1n.png

 

So i searched and read this thread where i think L.Spiro had same problem.

 

Then i searched my HLSLs programs to find where i could have NaNs.

Found one problem in my point light shader:

float4 posVS = gbPosition_tex.Sample(sampPointClamp, tc);
    // this has no effect !
	if (IsNAN(posVS))
		posVS = float4(0.0f, 0.0f, 0.0f, 1.0f);
	if (any(isinf(posVS)))
		posVS = float4(0.0f, 0.0f, 0.0f, 1.0f);

    ... //code skipped, nothing else touches posVS
	float3 v = normalize(-posVS.xyz);

     // but if i uncomment this there is NO more black squares ! 
	//if (IsNAN(v))
	//	v = float3(0.0f, 0.0f, 0.0f);
    ...// rest of lightning computation

Please read comments in code.

IsNAN is defined like this (i cant use "native" isnan function because i cant set /Gis option inside VS2013 for fxc compiler options):

bool IsNAN(float n)
{
    return (n < 0.0f || n > 0.0f || n == 0.0f) ? false : true;
}

bool IsNAN(float2 v)
{
    return (IsNAN(v.x) || IsNAN(v.y)) ? true : false;
}

bool IsNAN(float3 v)
{
    return (IsNAN(v.x) || IsNAN(v.y) || IsNAN(v.z)) ? true : false;
}

bool IsNAN(float4 v)
{
    return (IsNAN(v.x) || IsNAN(v.y) || IsNAN(v.z) || IsNAN(v.w)) ? true : false;
}

wtf is going on? 

Share this post


Link to post
Share on other sites
Advertisement

i just looked at the code a little closer:

posVS = float4(0.0f, 0.0f, 0.0f, 1.0f);
posVS == (0, 0, 0, 1) 
float3 v = normalize(-posVS.xyz);

normalize:

a = abs(sqrt(v.x * v.x + v.y * v.y + v.z * v.z));
a == 0;

v.x = 0/0;
v.y = 0/0;
v.z = 0/0;

v.x == NaN;
v.y == NaN;
v.z == NaN;

IsNAN(v) == true

basically, if posVS has 0 in all three x, y, and z before you try to normalize it, then you'll end up with nans

 

 

 

 

Share this post


Link to post
Share on other sites
22 minutes ago, belfegor said:

That gives me same results as my code.

There is a isnan() intrinsic also ! Also, their is compiler flags that may relax rules over how to deal with math and change the result sometimes. No matter what, normalize of 0 is a bad idea :)

Share this post


Link to post
Share on other sites

@iedoc

We don't know how normalize is implemented in HLSL. But still i thought about that before and i tried even with this :

if (IsNAN(posVS))
		posVS = float4(1.0f, 1.0f, 1.0f, 1.0f);
	if (any(isinf(posVS)))
		posVS = float4(1.0f, 1.0f, 1.0f, 1.0f);

rest of the code is the same, but same results.

 

@galop1n 

cant use native isnan:

warning X3577: value cannot be NaN, isnan() may not be necessary.  /Gis may force isnan() to be performed

CANT set /Gis option in VS2013 fxc compiler options

Share this post


Link to post
Share on other sites

but what if gbPosition_tex.Sample(sampPointClamp, tc) returns (0, 0, 0, 1) or something (black)?

Share this post


Link to post
Share on other sites
1 minute ago, belfegor said:

Thats it. 


float3 v = normalize(-(posVS.xyz + 0.0001f.xxx));

Thank you very much.

and what if posVS.xyz is -0.0001f ? You should probably do better with "float3 v = any(posVS!=0) ? normalize(posVS) : 0;" But how your position can be full zero in the first place ?

Share this post


Link to post
Share on other sites

"But how your position can be full zero in the first place ?"

Great question.Because i cleared that texture with 0's. I am thinking now what is better clear value to avoid future problems.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

Participate in the game development conversation and more when you create an account on GameDev.net!

Sign me up!