Very peculiar GLSL bug

Started by
6 comments, last by _the_phantom_ 18 years, 2 months ago
Hi, I'm trying to write some GLSL support into a game I'm working on so I can do PCF shadow maps. I have been able to resolve most of the problems that have come up, but this one really sticks me. I was trying to do shadow comparison in a fragment shader, but every time I include a call to shadow2D or shadow2DProj or whatnot, the shader fails to validate and will not execute. So, I looked over my shaders and couldn't see anything wrong. As a sanity check, I tried out the shaders from the PCF tutorial on GameTutorials.com. In that tutorial program the shader runs flawlessly. In my program the shader compiles, but fails to link because "Not all shaders have valid object code." I'm not exactly sure what that means. But the problem has to be in my own code. The root of the problem seems to be that either a shader will not validate or will not link if I call shadow2D, etc at any point. If I comment out calls to shadow2D, etc, then the shader will compile, link, validate, and run in my game provided there is nothing else wrong with it. Since other programs on my computer can handle shaders with these calls, the problem, again, must reside in my own code. But why shadow2D? Here is the code that I'm using to compile my shaders after I load them from file. Let me know if anything is being set up wrong. It's probably right in front of me...

bool Shader::compile(const char *vertexShader, const char *fragmentShader)
{
	int status=GL_FALSE;

	// Flush errors, error reports will concern the following calls only
	FLUSH_GL_ERROR();

// Create the shader objects
	unsigned int v = glCreateShader(GL_VERTEX_SHADER);
	unsigned int f = glCreateShader(GL_FRAGMENT_SHADER);	

// Send the source code to the driver
	glShaderSource(v, 1, (const char **)&vertexShader, NULL);
	glShaderSource(f, 1, (const char **)&fragmentShader, NULL);

// Compile the Vertex Shader
	glCompileShader(v);

	CHECK_GL_ERROR();
	glGetShaderiv(v, GL_COMPILE_STATUS, &status);
	if(status == GL_FALSE)
	{
		// Failed to compile the vertex shader
		printInfoLog(v);
		ASSERT(false, "Vertex Shader failed to compile! Check the error log for details.");
		return false;
	}

// Compile the Fragment Shader
	glCompileShader(f);

	CHECK_GL_ERROR();
	glGetShaderiv(f, GL_COMPILE_STATUS, &status);
	if(status == GL_FALSE)
	{
		// Failed to compile the fragment shader
		printInfoLog(f);
		ASSERT(false, "Fragment Shader failed to compile! Check the error log for details.");
		return false;
	}

	// Create a program name
	programHandle = glCreateProgram();
	CHECK_GL_ERROR();
	ASSERT(glIsProgram(programHandle)==GL_TRUE, "Shader::compile  ->  The program handle is not a valid handle to a GLSL shader program");

	// Attach object code to the program
	glAttachShader(programHandle, v);
	glAttachShader(programHandle, f);

	// Link the object code to finish the program
	glLinkProgram(programHandle);
	printInfoLog(programHandle);

	CHECK_GL_ERROR();
	glGetProgramiv(programHandle, GL_LINK_STATUS, &status);
	if(status == GL_FALSE)
	{
		// Failed to link correctly
		THROW("Failed to link shader");
		return false;
	}
	else
	{
		// Lastly, validate the shader to determine whether it will run at all
#ifdef _DEBUG
		glValidateProgram(programHandle);
		glGetProgramiv(programHandle, GL_VALIDATE_STATUS, &status);
		ASSERT(status==GL_TRUE, "Shader::compile  ->  The GLSL shader program could not be validated, it is garaunteed to not execute.");
#endif

		// Get the locations of the uniforms samplers for each texture unit
		for(int unit=0; unit<16; ++unit)
		{
			char buffer[32]={0};
			sprintf(buffer, "tex%i", unit);

			// If a uniform could not be found, then -1 will bestored
			texLocs[unit] = getLocation(buffer);
		}
	}

	return true;
}


-Andrew
Advertisement
can you post the shader code as well.. (both VS and FS)
This is a shadow map shader from gametutorials.com, however, I renamed the sampler to tex1. Its not the PCF one I mentioned above, but it suffers the same error when I execute it in my program. The demo program on gametutorials.com has no problems, but in my program, while compiling the vertex shader and fragment shader causes no error, it fails to link because "not all shaders have valid object code."

Vertex Shader:
varying vec4 projCoord;void main(){	vec4 realPos = gl_ModelViewMatrix * gl_Vertex;  	projCoord = gl_TextureMatrix[0] * realPos;	gl_FrontColor = gl_Color;	gl_Position = ftransform();}


Fragment Shader:
uniform sampler2DShadow tex1;varying vec4 projCoord;void main (){	const float kTransparency = 0.3;	vec4 color = gl_Color;	float rValue = shadow2DProj(tex1, projCoord).r + kTransparency;	rValue = clamp(rValue, 0.0, 1.0);	vec3 coordPos  = projCoord.xyz / projCoord.w;	if(coordPos.x >= 0.0 && coordPos.y >= 0.0 && coordPos.x <= 1.0 && coordPos.y <= 1.0 )   	{		gl_FragColor = color * rValue;	}	else	{		gl_FragColor = color;	}}


--------------------------------------------------------

I tried to create the simplest shader possible that still causes an error. The following shader compiles, links, but fails to validate.
Vertex Shader:
varying vec2 Texcoord;varying vec4 projCoord;void main(){	    // Pass down the texcoord for this vertex    Texcoord = gl_MultiTexCoord0.xy;        // Pass down the projected shadow coord    vec4 realPos = gl_ModelViewMatrix * gl_Vertex;  	projCoord = gl_TextureMatrix[1] * realPos;        // Transform vertex	gl_Position = ftransform();} 

Fragment Shader:
uniform sampler2D tex0;uniform sampler2DShadow tex1;varying vec2 Texcoord;varying vec4 projCoord;void main(){		vec4 textureColor = texture2D(tex0, Texcoord);	vec4 shadowColor = shadow2DProj(tex1, projCoord);	gl_FragColor = textureColor * shadowColor;}


-------------------------------

I was surprised to find that this shader compiles, links, validates, and runs. However, it does not behave correctly.

Vertex Shader:
varying vec4 projCoord;void main(){	vec4 realPos = gl_ModelViewMatrix * gl_Vertex;  	projCoord = gl_TextureMatrix[0] * realPos;		gl_Position = ftransform();}


Fragment Shader:
uniform sampler2DShadow tex1;varying vec4 projCoord;void main (){	float rValue = shadow2DProj(tex1, projCoord).r;	gl_FragColor = vec4(rValue);}


I would expect that if it were to run, it would color each fragment according to the shadow value at that point. Instead, it appears to project the base map texture, which is not even read in this shader. The fragment shader above behaves identically to this other fragment shader:

uniform sampler2D tex0;varying vec4 projCoord;void main (){	float rValue = texture2DProj(tex0, projCoord).r;	gl_FragColor = vec4(rValue);}


---------------------------------

If it affects anything:

* Every time an object is about to be drawn, my program attempts to bind texture unit 0 to a uniform called tex0, texture unit 1 to tex1, texture unit 2 to tex2, etc.

* Each object has its own shadow map and FFP shadows work perfectly. Right now I'm only trying to cast the shadow from one object with these shaders.

---------------------------------

-Thank you

EDIT: sloppy copy/paste

[Edited by - Foxostro on January 29, 2006 1:19:27 PM]
Got nothing? :(

Am I building my shaders correctly? It looks good to me, but I might be missing something.

Could some other GL state that I have set elsewhere in the program interfere with GLSL?
SOLVED

glValidateProgram lies.
If I ignore the fact that "If validation is successful, program is guaranteed to execute given the current state. Otherwise, program is guaranteed to not execute." and try to execute the shader anyway, then it will execute.


When there are "if" statements, the shader may compile with no errors, but they will cause it to fail to link. Does this mean my Radeon 9800XT doesn't support branching statements?


Quote:
I would expect that if it were to run, it would color each fragment according to the shadow value at that point. Instead, it appears to project the base map texture, which is not even read in this shader. The fragment shader above behaves identically to this other fragment shader [...]

I was setting a sampler-uniform wrong for that particular test.
Quote:Original post by Foxostro
When there are "if" statements, the shader may compile with no errors, but they will cause it to fail to link. Does this mean my Radeon 9800XT doesn't support branching statements?


As far as I know, dynamic branching is only supported on shader model 3 hardware...
But you would think it would give you a more descriptive error when I used an "if" :b Its not even that I can't use "if" ever, I can use it when comparing a float against a compile time constant or a literal constant. I do that twice in my phong shader...

And I still don't get glValidateProgram :/
glValidateProgram doesnt lie, it does precisly what it is ment todo, you are just failing to take into account a certain fact : it returns true or false depending on the whole OpenGL state at the time of calling.

Now, you are using a shadow lookup, however the GLSL state tracking at that point says that texture unit 0 is a standard texture unit and not in shadow mapping mode, thus the outcome of the shader may or may not be valid, therefore glValidateProgram returns false, as it is ment todo.

This topic is closed to new replies.

Advertisement