Sign in to follow this  
JohanLarsson

GLSL problem - saving data to alpha channel

Recommended Posts

Hello all.

I'm trying to use a GLSL shader to transform a texture and place it in another one. It mostly works - however, if I try to save any value that is below or equal to 0.1 in the alpha channel of any pixel of texture, no data at all seems to get written to the texture. Very weird. If I ensure that the data I write to the alpha channel is always at least .101, everything works as expected (except for me losing valuable precision in my data).

Anyone got an idea as to why this happens?

Here's the shader. Note the "special" variable at the end. Just putting vertical.y in the alpha channel breaks the entire texture:
[code]#version 130
uniform sampler2D tex;
out vec4 outColor;
void main() {
float u0 = (gl_TexCoord[0].x * 2) - 1;
float v0 = (gl_TexCoord[0].y * 2) - 1;

v0 = v0 * abs(u0);
v0 = (v0 + 1) * 0.5;
vec2 newCoords = vec2(gl_TexCoord[0].x, v0);

vec4 horizontal = texture2D(tex, newCoords);
vec4 vertical = texture2D(tex, newCoords.yx);
// TODO: WHY DOES vertical.y NOT WORK? Why does it seem alpha is so special?
float special = vertical.y;
if (special < 0.101)
special = 0.101;
outColor = vec4(horizontal.x, horizontal.y, vertical.x, special); }[/code]


I'm not doing anything particularly exotic when rendering the shader. The calling code looks like this:
[code]

GL.Disable(EnableCap.Blend)
// The target FBO for rendering
GL.BindFramebuffer(FramebufferTarget.Framebuffer, ShadowBuffer)
GL.Viewport(0, 0, res, res)
// The target texture attachment in the FBO

GL.DrawBuffer(DrawBufferMode.ColorAttachment1)
GL.ClearColor(0.f, 0.f, 0.f, 0.f)
GL.Clear(ClearBufferMask.ColorBufferBit ||| ClearBufferMask.DepthBufferBit)
GL.LoadIdentity()

GL.BindTexture(TextureTarget.Texture2D, ShadowTexture)
// This is the shader program in question
GL.UseProgram(DistortProgram)
GL.Begin(BeginMode.Quads)
GL.TexCoord2(0., 0.)
GL.Vertex2(-0.5, 0.5)
GL.TexCoord2(0., 1.)
GL.Vertex2(-0.5, -0.5)
GL.TexCoord2(1., 1.)
GL.Vertex2(0.5, -0.5)
GL.TexCoord2(1., 0.)
GL.Vertex2(0.5, 0.5)
GL.End()
GL.UseProgram(0)
[/code]

The code where I generate the FBO and target texture attachment follows below:
[code]

GL.BindTexture(TextureTarget.Texture2D, DistortTexture)
GL.TexParameter( TextureTarget.Texture2D, TextureParameterName.TextureMinFilter, (int) TextureMinFilter.Nearest)
GL.TexParameter( TextureTarget.Texture2D, TextureParameterName.TextureMagFilter, (int) TextureMagFilter.Nearest)
GL.TexParameter( TextureTarget.Texture2D, TextureParameterName.TextureWrapS, (int) TextureWrapMode.Clamp)
GL.TexParameter( TextureTarget.Texture2D, TextureParameterName.TextureWrapT, (int) TextureWrapMode.Clamp)
GL.TexImage2D(TextureTarget.Texture2D, 0, PixelInternalFormat.Rgba8, res, res, 0, PixelFormat.Rgba, PixelType.UnsignedByte, IntPtr.Zero)


GL.GenFramebuffers(1, &ShadowBuffer)
GL.BindFramebuffer(FramebufferTarget.Framebuffer, ShadowBuffer)
GL.FramebufferTexture2D(FramebufferTarget.Framebuffer, FramebufferAttachment.ColorAttachment0, TextureTarget.Texture2D, ShadowTexture, 0)
GL.FramebufferTexture2D(FramebufferTarget.Framebuffer, FramebufferAttachment.ColorAttachment1, TextureTarget.Texture2D, DistortTexture, 0)
[/code]

Any help is appreciated.

Share this post


Link to post
Share on other sites
You may be seeing an optimization called "alpha testing". Don't ask me how to disable it in OpenGL - I'd venture a guess that it is possible though.

In normal rendering, low alpha values of a pixel (when alpha blending) cause the pixel's contribution to the final image to be low, so the rendering can be sped up by not blending the pixels that have an alpha value less than a given treshold. High quality optimization level would simply set the treshold to the minimum alpha value representable on the render target, so in effect all meaningful pixels would still be written and only the actually invisible ones discarded. In cases like yours, though, it would be the intended behavior to write the pixels regardless of whether they pass a visual treshold or not.

In D3D, the treshold can be set by device states directly, but I don't have enough expertise on OpenGL to say how to set an equivalent state. However, I hope that I pointed you to the right direction. I guess googling for "alpha test opengl" would yield some results.

Share this post


Link to post
Share on other sites
Hmm, I tried adding GL.Disable(EableCap.AlphaTest), but that didn't help.

However, I just now noticed that rendering with the shader program results in an Invalid Operation error if I try removing the special handling of the alpha channel. Still don't know why.

Share this post


Link to post
Share on other sites
This doesn't work, and results in no usable data when rendering the texture:
[code]

#version 130
uniform sampler2D tex;
out vec4 outColor;
void main() {
//translate u and v into [-1 , 1] domain
float u0 = (gl_TexCoord[0].x * 2) - 1;
float v0 = (gl_TexCoord[0].y * 2) - 1;

//then, as u0 approaches 0 (the center), v should also approach 0
v0 = v0 * abs(u0);
//convert back from [-1,1] domain to [0,1] domain
v0 = (v0 + 1) * 0.5;
//we now have the coordinates for reading from the initial image
vec2 newCoords = vec2(gl_TexCoord[0].x, v0);

//read for both horizontal and vertical direction and store them in separate channels
vec4 horizontal = texture2D(tex, newCoords);
vec4 vertical = texture2D(tex, newCoords.yx);
// TODO: WHY DOES vertical.y NOT WORK? Why does it seem alpha is so special?
float special = vertical.y;
outColor = vec4(horizontal.x, horizontal.y, vertical.x, special);
}
[/code]
This, however, works:
[code]
#version 130
uniform sampler2D tex;
out vec4 outColor;
void main() {
//translate u and v into [-1 , 1] domain
float u0 = (gl_TexCoord[0].x * 2) - 1;
float v0 = (gl_TexCoord[0].y * 2) - 1;

//then, as u0 approaches 0 (the center), v should also approach 0
v0 = v0 * abs(u0);
//convert back from [-1,1] domain to [0,1] domain
v0 = (v0 + 1) * 0.5;
//we now have the coordinates for reading from the initial image
vec2 newCoords = vec2(gl_TexCoord[0].x, v0);

//read for both horizontal and vertical direction and store them in separate channels
vec4 horizontal = texture2D(tex, newCoords);
vec4 vertical = texture2D(tex, newCoords.yx);
// TODO: WHY DOES vertical.y NOT WORK? Why does it seem alpha is so special?
float special = vertical.y;

special = max(special, 0.101);
outColor = vec4(horizontal.x, horizontal.y, vertical.x, special);
}[/code]

The only difference is the second to last line.

I suspect I'm invoking undefined behaviour at some point in the shader, but I can't for the life of me see it (and OpenGL reports no errors when compiling the shader).

Share this post


Link to post
Share on other sites
I don't spot any obvious errors. I'm more used to HLSL (D3D shader language), but I assume that the data type semantics are very similar here.

If it is indeed the shader that causes errors, I would be ready to blame the driver's shader compiler just about now. The shader code itself is not even that complex.

Have you tried "max (special, 0.0)" ?

Share this post


Link to post
Share on other sites
[quote name='JohanTL' timestamp='1318009259' post='4870197']
Hmm, I tried adding GL.Disable(EableCap.AlphaTest), but that didn't help.

However, I just now noticed that rendering with the shader program results in an Invalid Operation error if I try removing the special handling of the alpha channel. Still don't know why.
[/quote]
Weird, it should work...
You may also try to play with alpha function and cutoff level

[url="http://www.opengl.org/sdk/docs/man/xhtml/glAlphaFunc.xml"]http://www.opengl.or...glAlphaFunc.xml[/url]

for example glAlphaFunc (GL_ALWAYS, 0)
Other than that, it may be bug in your driver - you may try to rearrange the code a bit - for example, instead of newCoords.yx copy the values to new vec2 and shufle manually. You may also try to test it by adding some extra shader instructions (like multiplication with 0.9999) here or there to see, if it goes away.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this