Sign in to follow this  
Nathan Baum

A GLSL problem

Recommended Posts

Nathan Baum    1027
So I'm having weird intermittent problems with shaders. Simply put, sometimes the shader fails to execute and the result is random nonsense (it _doesn't_ fall back on the fixed functionality pipeline). glGetInfoLogARB says everything is fine with the shader. I'm using an ATI 9550 with 256MB of VRAM, on a Linux system with version 8.23.7 of ATI's X drivers. Fragment shader:
varying vec3 normal;
varying vec3 lightDir;

uniform sampler2D terrain_texture;

const float sealevel = 0.5;

float height (vec2 texCoord)
{
  return max(texture2D(terrain_texture, texCoord).r, sealevel - 0.01);
}

float inshadow (vec2 pos)
{
  const float scale = 0.05;
  float h = height(pos.st);
  for (int i = 0; i < 1; ++i) {
    pos += normalize(lightDir).xy * scale;
    //if (h < height(pos.st))
    //  return 1.0;
  }
  return 0.0;
}

vec3 sample_normal (vec2 texCoord)
{
  vec3 n;
  n = vec3(0.0, 0.0, 1.0);
  n = vec3(0.0, 0.0, 0.025);
  float h = height(texCoord);
  float u = h - height(texCoord + vec2( 0.0, -0.002));
  float d = h - height(texCoord + vec2( 0.0,  0.002));
  float l = h - height(texCoord + vec2(-0.002, 0.0));
  float r = h - height(texCoord + vec2( 0.002, 0.0));
  n = n + vec3( 0.0, -1.0, 0.0) * (d - u);
  n = n + vec3(-1.0,  0.0, 0.0) * (l - r);
  n = normalize(n);
  return n;
}

vec4 light (vec3 normal)
{
  vec3 n, halfV, viewV, ldir;
  float NdotL, NdotHV;
  vec4 color;
  n = normalize(normal);
  NdotL = max(dot(n, normalize(lightDir)), 0.0);
  if (NdotL > 0.0) {
    color = vec4(1.0, 1.0, 1.0, 1.0) * NdotL;
  }
  return vec4(color.rgb, 1.0);
}

vec3 land (float phase)
{
  if (phase < 0.01)
    return vec3(1.0, 1.0, 0.0);
  if (phase < 0.5) {
    phase = phase / 0.5;
    return (vec3(0.0, 0.5, 0.0) * (1.0 - phase) +
	    vec3(0.5, 0.5, 0.0) * phase);
  }
  if (phase < 0.7)
    return vec3(0.5, 0.5, 0.5);
  return vec3(1.0, 1.0, 1.0);
}

void main()
{
  vec4 color;
  vec2 pos = gl_TexCoord[0].st;
  float h = height(pos.st);
  color = vec4(h, h, h, 1.0);
  if (h < sealevel) {
    color.rgb = vec3(0.0, 0.0, 0.7);
  } else {
    color.rgb = land(smoothstep(sealevel, 1.0, color.r));
  }
  //color *= mix(light(sample_normal(pos.st)) * inshadow(pos), vec4(1.0), (1.0 - h));
  color *= mix(light(sample_normal(pos.st)), vec4(1.0), (1.0 - h));
  gl_FragColor = color;
}


Vertex shader:
varying vec3 lightDir;
	
void main()
{	
  lightDir = vec3(gl_LightSource[0].position - gl_ModelViewMatrix * gl_Vertex);
  lightDir.y = -lightDir.y;
  gl_TexCoord[0] = gl_MultiTexCoord0;
  gl_Position = ftransform();
} 


In this example, the fault occurs when I uncomment the two commented lines in the inshadow function. Yes, I am aware the code doesn't call that function. Which makes the bug doubly weird. It also occurs with some other code. There is no pattern that I can see. At first I thought I was hitting some kind of instruction limit, but the above code is quite a bit shorter than previous incarnations. I also thought it was a limit on execution time, but it obviously can't be since the code causing the fault is commented out. Anyone got any clues? (The texture I'm rendering with is a simple grayscale Perlin noise. Yes, I know I should be precalculating the normal map on the CPU, and I intend to do so later. But that can't be what's causing the problem.) Edit: I've solved this problem by moving the return out of the loop. However, since the code was never called, that doesn't seem to be a sufficient explanation for what was wrong. This has happened before without returning from inside a loop, so this doesn't help me avoid the problem in the future.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this