# GLSL stencil z-pass/z-fail controlled in shader.

This topic is 2441 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

hello everyone, i'm attempting to do shadow volume via edge counting outline here: http://hungryspoon.com/PX_web/paper/paper.pdf via shaders. the problem is how to do counting in the shader across vertex's that have been identified as part of a specific edge.

i've come to the idea of using a framebuffer in an initial pass to generate the edge counting, but the problem with that is that i have no way of reading the framebuffer's current value, and to increment/decrement a specific value, so i was thinking of using the stencil INCR_WRAP/DECR_WRAP in the stencil buffer to do the counting.

so, i have a few questions:

1. in a fragment shader, can i specify the specific pixel for the fragment to write to?
2. in a fragment shader, can i control z-pass/z-fail case for the stencil buffer?
3. can i bind a sampler that is the framebuffer's stencil layer?

##### Share on other sites
Hi!

1. in a fragment shader, can i specify the specific pixel for the fragment to write to?

Yes, but only since OpenGL 4.2.
Anything before GL 4.2 needs scattering. In this paradigm you write the target position to a FBO. Subsequently you render a vertex grid (GL_POINTS) with one vertex per pixel, fetch in the vertex shader the target position and transform the vertex to that position.

2. in a fragment shader, can i control z-pass/z-fail case for the stencil buffer?

You mean changing the stencil behavior in the shader? Again, I'd refer you to GL 4.2, since it gives you random access writes and with this the possibility to modify a value in a buffer as you wish. If GL 4.2 is no option for you, you could try to use two FBOs in a ping-pong fashion to emulate some sort of stencil buffer. You could read from the source FBO (i.e. its texture), decide in the fragment shader what to do with it and write the result to the destination FBO. In the end of the frame you swap both FBOs.

3. can i bind a sampler that is the framebuffer's stencil layer?

Yes, I think so, since Direct3D can do it. You probably find something in the specs.

If you want to create shadow volumes with shaders then rather consider using a geometry shader with adjacency information to detect edges. To my knowledge this is the most common practice.

##### Share on other sites
hey Tsus, thanks for the reply=-).

allow me to clarify a few things.
for #1, i did consider using GL_POINTS to do specific fbo writes, i just wasn't certain if this would be an accurate method for ensuring perfectly writing to the correct locations.

for #2, i don't want to change the behavior defined by glStencilOp, i simply want to ensure which behavior is done, for example, if i do a "discard" in the fragment shader, well that trigger the z-fail set by glStencilOp?

and for #3, i'll check into it.

thanks again, i am using a geometry shader, but i want to do it without having to supply adjacency information, which is why i'm seeking a method that allows me to do edge counting accurately, thus eliminating the extra adjacency info, instead requiring a pre-pass to generate a sampler to be used by another shader which well create the actual shadow.

##### Share on other sites
Hey!

for #1, i did consider using GL_POINTS to do specific fbo writes, i just wasn't certain if this would be an accurate method for ensuring perfectly writing to the correct locations.

It should work, if you compute the position correctly.
As a small example, here is a vertex shader I used for creating histograms (the fragment shader just outputs a 1.0/255.0). The shader accumulates in the bottom row of the back buffer (quite dirty ).
uniform sampler2D imageTexture; uniform float SCREEN_WIDTH; // = 512.0; void main() { // read color vec3 pixelColor = texture2D(imageTexture, gl_MultiTexCoord0.st).rgb; // compute average gray value float grayValue = 0.333333 * (pixelColor.r + pixelColor.g + pixelColor.b); // compute x-position. The target pixel is somewhere between (0,0) and (255,0) float posX = grayValue / SCREEN_WIDTH * 255.0; // map the position in [0,1] to the interval [-1,1]. gl_Position = vec4(2*posX-1, -1, 0, 1); }

for #2, i don't want to change the behavior defined by glStencilOp, i simply want to ensure which behavior is done, for example, if i do a "discard" in the fragment shader, well that trigger the z-fail set by glStencilOp?

Influencing the stencil fail seems difficult. Depth fail on the other hand should be possible, though perhaps a discard won't trigger it, since technically it has nothing to do with the depth. Maybe it works if you change the depth value, such that it gets clipped?
For a z-fail:
gl_FragDepth = 2; // definitely outside.
For a z-pass:
gl_FragDepth = 0; // always true if glDepthFunc is GL_LEQUAL.

As a side note, GL 4.2 also allows you to do random access writes to a buffer in the geometry shader.

thanks again, i am using a geometry shader, but i want to do it without having to supply adjacency information, which is why i'm seeking a method that allows me to do edge counting accurately, thus eliminating the extra adjacency info, instead requiring a pre-pass to generate a sampler to be used by another shader which well create the actual shadow.

Alright, I see. Btw, what’s the reason for not generating adjacency information? I think it’s probably faster, since it requires only one pass (the main work is done in a pre-process).

Mmm. How do you determine the ID of the edge (to identify which counter to increase)?

Cheers!

##### Share on other sites

Hey!

[quote name='slicer4ever' timestamp='1334114274' post='4930094']
for #1, i did consider using GL_POINTS to do specific fbo writes, i just wasn't certain if this would be an accurate method for ensuring perfectly writing to the correct locations.

It should work, if you compute the position correctly.
As a small example, here is a vertex shader I used for creating histograms (the fragment shader just outputs a 1.0/255.0). The shader accumulates in the bottom row of the back buffer (quite dirty ).
uniform sampler2D imageTexture; uniform float SCREEN_WIDTH; // = 512.0; void main() { // read color vec3 pixelColor = texture2D(imageTexture, gl_MultiTexCoord0.st).rgb; // compute average gray value float grayValue = 0.333333 * (pixelColor.r + pixelColor.g + pixelColor.b); // compute x-position. The target pixel is somewhere between (0,0) and (255,0) float posX = grayValue / SCREEN_WIDTH * 255.0; // map the position in [0,1] to the interval [-1,1]. gl_Position = vec4(2*posX-1, -1, 0, 1); }
[/quote]
ah, awesome, so mapping points is accurate, thanks for the info=-):

[quote name='slicer4ever' timestamp='1334114274' post='4930094']
for #2, i don't want to change the behavior defined by glStencilOp, i simply want to ensure which behavior is done, for example, if i do a "discard" in the fragment shader, well that trigger the z-fail set by glStencilOp?

Influencing the stencil fail seems difficult. Depth fail on the other hand should be possible, though perhaps a discard won't trigger it, since technically it has nothing to do with the depth. Maybe it works if you change the depth value, such that it gets clipped?
For a z-fail:
gl_FragDepth = 2; // definitely outside.
For a z-pass:
gl_FragDepth = 0; // always true if glDepthFunc is GL_LEQUAL.

As a side note, GL 4.2 also allows you to do random access writes to a buffer in the geometry shader.
[/quote]
the problem with doing random access writes, is i need to also do random reads as well, i only need to increment/decrement a value by one, but without a way of knowing the value that's their already, i'm just going to be overwriting it. which is why i'm trying to use the stencils GL_INCR/GL_DECR to do my counting.

[quote name='slicer4ever' timestamp='1334114274' post='4930094']
thanks again, i am using a geometry shader, but i want to do it without having to supply adjacency information, which is why i'm seeking a method that allows me to do edge counting accurately, thus eliminating the extra adjacency info, instead requiring a pre-pass to generate a sampler to be used by another shader which well create the actual shadow.

Alright, I see. Btw, what’s the reason for not generating adjacency information? I think it’s probably faster, since it requires only one pass (the main work is done in a pre-process).

Mmm. How do you determine the ID of the edge (to identify which counter to increase)?

Cheers!
[/quote]

the reason i'm not generating adjacency info is because when i originally wrote my mesh containers(and alot of the animation/skeletal data), i had never used geometry shaders before, and unfortuantly i didn't provide any mechanism's to distringuish between adjacent/real data. the next iteration of the engine well support everything that i've learned with geometry shaders, but at the moment, i'm hoping to find an application side solution to my problem, rather than spending a few more weeks re-working the core concepts of my engine.

As for determining the ID of the edge, that's done at initialization, I pre-process each vertex with what edge it's attached to, since a triangle contains 3 edges, and 3 vertices, then each vertice should represent that particular edge(which in ine pre-process is determined to when that unique edge was found).

##### Share on other sites

[quote name='Tsus' timestamp='1334129408' post='4930159']
As a side note, GL 4.2 also allows you to do random access writes to a buffer in the geometry shader.

the problem with doing random access writes, is i need to also do random reads as well, i only need to increment/decrement a value by one, but without a way of knowing the value that's their already, i'm just going to be overwriting it. which is why i'm trying to use the stencils GL_INCR/GL_DECR to do my counting.
[/quote]
You have random access reads as well. (I didn't mention it, since it's nothing special, because we have them since there were textures.)
In summary, you can read and write from the same buffer in GL 4.2 -- both with random access.

I can dig through my code and search for an old OIT demo I wrote, if you want. It's using atomic operations to increase a counter, which could be an option for you.

##### Share on other sites

[quote name='slicer4ever' timestamp='1334160770' post='4930290']
[quote name='Tsus' timestamp='1334129408' post='4930159']
As a side note, GL 4.2 also allows you to do random access writes to a buffer in the geometry shader.

the problem with doing random access writes, is i need to also do random reads as well, i only need to increment/decrement a value by one, but without a way of knowing the value that's their already, i'm just going to be overwriting it. which is why i'm trying to use the stencils GL_INCR/GL_DECR to do my counting.
[/quote]
You have random access reads as well. (I didn't mention it, since it's nothing special, because we have them since there were textures.)
In summary, you can read and write from the same buffer in GL 4.2 -- both with random access.

I can dig through my code and search for an old OIT demo I wrote, if you want. It's using atomic operations to increase a counter, which could be an option for you.
[/quote]

ah, awesome, i guess the last problem is that it's 4.2, which means very modern graphics cards are required(perhaps an extension for earlier implementation is available.)

also, this is exactly what i was looking for, and makes my idea of using a pre-pass completely unnecessary.

i've been searching for awhile for reading/writing to a buffer, and couldn't find a thing on it, this would be incredibly helpful, is their a special word for what the buffer is called that allows read/writes?

Edit: ok, so, in the meantime, i've been continuing work on my framebuffer idea, and have been googling for several hours for a solution. my problem is that I can not seem to attach to the framebuffer GL_STENCIL_ATTACHMENT by itself. I am attempting to create the stencil texture, but i can't seem to find a format/internalformat to use for the stencil buffer.

any help on this front is also much appreciated.

##### Share on other sites
Hi!

The OGL sample pack contains a sample for buffers that allow atomic operations (ogl-420-atomic-counter.cpp).
(That's where I learned it.)

Aside from that, I found my old code and paste you the important parts here.
This is how you create a buffer that allows atomic operations.
glGenBuffers(1, &bufCounter); glBindBuffer(GL_ATOMIC_COUNTER_BUFFER, bufCounter); glBufferData(GL_ATOMIC_COUNTER_BUFFER, sizeof(GLuint), NULL, GL_DYNAMIC_DRAW); glBindBuffer(GL_ATOMIC_COUNTER_BUFFER, 0);

Textures with random access writes are created just like any other texture.
glGenTextures(1, &bufStartOffset); glBindTexture(GL_TEXTURE_2D, bufStartOffset); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); glTexStorage2D(GL_TEXTURE_2D, 1, GL_R32UI, width, height);

Atomic Buffers and read/write textures are bound to the pipeline like this:
glBindBufferBase(GL_ATOMIC_COUNTER_BUFFER, 0, bufCounter); glBindImageTexture(1, bufFragmentPool, 0, GL_FALSE, 0, GL_READ_WRITE, GL_R32UI); glBindImageTexture(2, bufStartOffset, 0, GL_FALSE, 0, GL_READ_WRITE, GL_R32UI);

And here we have a fragment shader for order-independent transparency by fragment linked lists. More precisely, this is only the construction of the linked lists.
#version 420 core in vec3 normal, lightDir; layout(binding = 0, offset = 0) uniform atomic_uint Counter; layout(binding = 1, r32ui) uniform uimage2D FragmentPool; layout(binding = 2, r32ui) uniform uimage2D StartOffset; ivec2 getaddr(uint next) { return ivec2(next % 4096, next / 4096); } void main() { vec3 n = normalize(normal); float NdotL = abs(dot(n, normalize(lightDir))); vec4 color = vec4(vec3(1,1,0.3) * NdotL, 0.4f); // Increment and get current pixel count. uint nPixelCount= atomicCounterIncrement(Counter); // Read and update Start Offset Buffer. (Retrieve the head of the linked list.) uint nOldStartOffset = imageAtomicExchange(StartOffset, ivec2(gl_FragCoord.xy), nPixelCount); // Store fragment link (color, depth and offset to next fragment in the linked list). imageStore(FragmentPool, getaddr(nPixelCount * 3) , uvec4(packUnorm4x8(color))); imageStore(FragmentPool, getaddr(nPixelCount * 3 + 1) , uvec4(floatBitsToUint(gl_FragCoord.z))); imageStore(FragmentPool, getaddr(nPixelCount * 3 + 2) , uvec4(nOldStartOffset)); }
As you can see it involves some conversions and packing. I figured you might want to see this for future reference, too.

Cheers!

##### Share on other sites
thanks alot mate, you are as helpful as ever=-)

1. 1
2. 2
Rutin
19
3. 3
khawk
18
4. 4
A4L
14
5. 5

• 12
• 16
• 26
• 10
• 44
• ### Forum Statistics

• Total Topics
633767
• Total Posts
3013734
×