Sign in to follow this  

GLSL NaN problem

This topic is 3459 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I just got hit by this recently when I improved an incorrect down-sampling in my tone-mapper. Suddenly at times the screen gets totally black and never recovers from it. I track this down to the tone mapper receiving an NaN value as the average scene key. This I tracked down to be colors with all components set to NaN at the beginning of the down sample stage. Hence whenever the GLSL shaders during rendering spit out even one NaN value the entire down sampling is doomed. Now comes though the funny part: How in gods name can an nVidia card spit out a color with all NaN?. I need to get this fixed so I can at last go on and worry about this driver problem later on. Is there a way I can detect an NaN value in the shader and replace it with something like vec4(0) or such a thing to avoid a total black-out?

Share this post


Link to post
Share on other sites
Quote:
Original post by RPTDHow in gods name can an nVidia card spit out a color with all NaN?

http://steve.hollasch.net/cgindex/coding/ieeefloat.html
Most likely you're dividing by 0 or something silly like that.

As for detecting, the standard way I know of is to compare it with itself and see if it evaluates to false.
According to the IEEE standard, NaN != NaN.

But GPU's generally don't implement the complete IEEE standard, so I don't know if that'll work in your case.

But you're probably better off finding out why you end up with NaN's in the first place, than trying to hack around it like this.

Share this post


Link to post
Share on other sites
Looks like an nVidia driver bug. I tracked the NaN down to some shader being executed and this tracks down to some normal being calculated beforehand. I did all the precautions like testing for 0-length before normalizing and min(*,65504) to prevent overflow but yet there creep in a handful ( up to 10 ) of NaN values during calculation.

Anybody has experience with bugged normalize call in GLSL on nVidia machines? What version should be safe of this driver bug?

Share this post


Link to post
Share on other sites

This topic is 3459 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this