Sign in to follow this  
svnstrk

GLSL problem integer and float comparison

Recommended Posts

hi,

im having problem trying to figure out what happened to my code. i notice this very significant error. here is the whole code: http://pastebin.com/XERH0NNx

and i want to point out this line:

int startPointValue = (int)startPointValuev4.r;
if(startPointValue*1.0 == (19049758.0+0.5) ) color = vec4(0.0, 0.0, 1.0, 1.0);

dont mind the complex number, i just want to figure out the 0.5 part which makes it a float. and startPointValue*1.0 is a float too, but since it was an integer, it does not suppose to has a decimal value.

when i draw the glsl, it shows a blue spot on the screen which means there are a time where startPointValue*1.0 == (19049758.0+0.5). And It shouldnt, right?

Did I miss something? its a basic math, did i do anything wrong on my code?


thanks in advance

Share this post


Link to post
Share on other sites
You cannot typecast in glsl. If you want to convert a float to an int you have to use a constructor.

Good:
float f = 0.5;
int I = int(f);

Bad:
float f = 0.5;
int I = (int)f;

The latter isn't legal syntax. Are you checking for warnings and errors when you compile? I don't really get what the rest of your problem is so this may or may not be your problem, just wanted to point that out.

Share this post


Link to post
Share on other sites
done that too, startPointValuev4.r is already an integer anyway, i just try to make sure by do a redundancy casting. im not sure what is the error too. no warning when compiling the GLSL.

im dealing with massive array of integer (tens of millions), and while the half of the array seems behaved correctly, the other half is not. this is one of the other messed up half that behaved weirdly.

any1 got any idea on this?

Share this post


Link to post
Share on other sites
I tweaked your code on pastebin, reposted here for posterity. I think that should work now. GLSL compiler doesn't do implicit casting like a C++ one, so be very careful - trace your types line by line if you ever encounter these errors.


#extension GL_EXT_gpu_shader4 : require
#extension GL_EXT_geometry_shader4 : enable
#extension GL_EXT_texture_integer : enable

uniform int depth;
varying float sphIndex;

uniform sampler3D tex;
uniform isampler3D texIndex;
uniform isampler2D texIndexStartPoint;

void main()
{
ivec2 vertCoord;

vec4 color = vec4(1.0, 1.0, 1.0, 1.0);
vec4 lastColor;

vertCoord.s = int(gl_TexCoord[0].s*300.0);
vertCoord.t = int(gl_TexCoord[0].t*300.0);

ivec4 startPointValuev4 = texelFetch(texIndexStartPoint, vertCoord, 0);
float startPointValue = float(startPointValuev4.r);


if( startPointValue*1.0 == (19049758.0+0.5) )
color = vec4(0.0, 0.0, 1.0, 1.0);

gl_FragColor = color;
}

Share this post


Link to post
Share on other sites
actually what i want to point out is the other way around.

startPointValuev4.r is an integer value. so it does not has decimal. but when i convert it to float, and compare it to a value with decimal, it gives me a true statement (which is from my code, i use draw blue pixel to check the result). im just confuse either this is a logic error or glsl are behaving weirdly when i put a massive interger texture. the problem is when im dealing with smaller texture, it behaves correctly.

Share this post


Link to post
Share on other sites
Quote:
19049758.0+0.5


If my math is correct this number is too large to get that precision with floating point. With a floating point number you only get ~7 digits of decimal precision.

In your case it's likely that 19049758.0+0.5 = 19049758, as the 0.5 is too small to be added to such a large number.

Yeah just checked here: http://babbage.cs.qc.edu/IEEE-754/Decimal.html

and it shows in FP32 that 19049758.0 and 19049758.5 both evaluate to 19049758.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this