# OpenGL Drawing alpha-tested foliage nicely (WARNING: image heavy)

This topic is 3827 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

Hey there. I came across this little trick a couple of weeks back which has been a god-send for me when rendering alpha-tested foliage. Basically it allows alpha-tested foliage to look as good as alpha-blended foliage without the sorting artifacts. I still see a lot of commercial and not-so-commercial games using straight ugly alpha-tested foliage that I figure I'm not the only one this could help. Anyway, ideally we want to draw our trees using alpha-blending so we get nice smooth edges. However, we need to draw the surfaces from farthest to closest or we get artifacts: So usually we use alpha-testing as we can't really afford the time to sort all those thousands of surfaces we're using to draw our trees with. Here's the same scene using alpha-testing with a cut-off of 0.5: As you can see the problem with alpha-tested foliage is the hard edges. The foliage tends to shimmer when moving as well. But at least it's simple and fast. This is what most games seem to use. We can combine both alpha-blending and alpha-testing in the same pass but then we get fringes which can be unsightly: Now here's the scene rendered using the trick I came across: As you can see, we get all the benefits of alpha-blended foliage with the simplicity of alpha-tested foliage, and sure looks nicer than either. Here's the trick (using OpenGL lingo). First render all your foliage using standard alpha-testing, but with a fairly high alpha cut-off:
    glBlendFunc( GL_ONE, GL_ZERO );
glEnable( GL_ALPHA_TEST );
glAlphaFunc( GL_GREATER, 0.75 );
...draw foliage...
Now redraw your foliage using alpha-blending but with a low alpha-test cut-off, depth-writes turned off, and depth testing set to less-than:
      glAlphaFunc( GL_GREATER, 8.0 / 255.0 );
glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );
glDepthFunc( GL_LESS );
...draw foliage again...
glDisable( GL_ALPHA_TEST );
glDepthFunc( GL_LEQUAL );
As you can see there is the cost of drawing all your foliage twice, but I think the image quality makes it worth it. You don't need to disable depth writing after each tree - draw all your organic foliage using the alpha-testing pass, then do it all again with the alpha-blending pass. Here's another comparison shot using a pine-tree (alpha-blended, alpha-tested, both): Static screenshots don't really do it justice. It looks nicer in motion. Anyway, hope that helps someone as it sure helped me. Apologies if this is a known trick. If that's the case, I figure there are some who don't know it.

##### Share on other sites
Nice pictures! Here's a link to a similar technique. Unfortunately the extra pass is always the first to go when I get around to optimizing because it uses up so much fill rate.

##### Share on other sites
Does anyone know if this method can be combined with the ones above? It's for billboards but might apply here.

##### Share on other sites
I saw a cool trick in GPU Gems 2 that dealt with the same problem, but I was just skimming in a bookstore and I don't remember what it was. Anyone have the book handy? It was right near the beginning -- I believe it was the grass chapter.

##### Share on other sites
The most robust solution I've seen is to use alpha-to-coverage which is a simple flag enable. It isn't perfect (it's effectively sub-pixel "screen-door" anti-aliasing) and needs to be used with MSAA, but it gives you N levels of free sorting per-pixel when N is the number of MSAA samples.

I've seen the technique in the OP before actually and it's usable, however it has several artifacts that you can actually see in the post. In particular note the left side of the top middle branch in the left-most image on the bottom row of images. (... Wow! Hopefully you can decode that.)

Still it may be preferable to typical alpha testing which looks even worse in many situations.

 The trick they use in Gems 2 is also a "screen-door" effect, but at the pixel level rather than sub-pixel so it won't look as good as alpha-to-coverage, and is more expensive to boot.

##### Share on other sites
I think the artifact you refer to are actually present on other images and using other technique, they seem to be texture wrapping artifact.

That is if you talk about the pixels in the air?

##### Share on other sites
Nice. In your second pass, would it not make a little more sense to render with alpha LEQUAL 0.75?
BTW, I think Tom Forsyth described this technique about 5 years ago on the DX or GA mailing list. As with anything computer graphics related, its pretty hard to invent something that hasn't been done before :-(

##### Share on other sites
So it was a known technique. Wish I'd found it before now. I definitely get the impression it's not a widely known trick because there are a lot of games that have ugly aliased alpha-tested foliage. But then, as jamesw says, that could simply be a case of not wanting to have to draw it all twice.
Quote:
 I've seen the technique in the OP before actually and it's usable, however it has several artifacts that you can actually see in the post. In particular note the left side of the top middle branch in the left-most image on the bottom row of images. (... Wow! Hopefully you can decode that.)

As Kern_d mentioned, it looks like a wrapping artifact. The models I used aren't the best.
Quote:
 Nice. In your second pass, would it not make a little more sense to render with alpha LEQUAL 0.75?

Give it a go. I haven't analysed the technique much, just enough to verify it works. I didn't invent the trick, just encountered it being used by an old game and thought I'd share it as it helped me a lot.
Quote:
 The most robust solution I've seen is to use alpha-to-coverage which is a simple flag enable.

I did play with alpha-to-coverage a while ago but wasn't very impressed. I can't recall if I used antialiasing with it or not. For me personally I found the trick I described here produced nicer results. I might go revisit a2c with antialiasing now though and see if it makes a difference.

##### Share on other sites
alpha-to-coverage doesnt even work at all without anti-aliasing--it is only activated if AA is enabled.

##### Share on other sites
Quote:
 Original post by PlayerXAs Kern_d mentioned, it looks like a wrapping artifact. The models I used aren't the best.

Fair enough, although I'm pretty sure artifacts could occur depending on the "threshold" values that you choose. At the very least banding near those thresholds. Now of course whether a degenerate case can be constructed (I'm almost positive that it can) and whether it comes up in practice are two different things :)

Quote:
 Original post by PlayerXI did play with alpha-to-coverage a while ago but wasn't very impressed. I can't recall if I used antialiasing with it or not. For me personally I found the trick I described here produced nicer results. I might go revisit a2c with antialiasing now though and see if it makes a difference.

Yeah A2C will do absolutely nothing without MSAA enabled since it relies on writing the alpha value to the MSAA sample "coverage", which in turn uses the MSAA hardware to store and sort the N samples, averaging them at the end (if you use a standard resolve... in D3D10 you could actually do whatever you want with the samples after the fact).

##### Share on other sites
Alpha-to-coverage seemed to work without AA, in that it drew the transparent areas using some kind of dithering. That's why I wasn't impressed. It definitely sounds like I didn't use it in its best light though, so will have to have another go.

##### Share on other sites
Quote:
 Original post by PlayerXAlpha-to-coverage seemed to work without AA, in that it drew the transparent areas using some kind of dithering. That's why I wasn't impressed. It definitely sounds like I didn't use it in its best light though, so will have to have another go.

Hmm interesting, I just assumed it would do nothing w/o the anti-aliasing hardware being active. In any case, sub-pixel screen-door should certainly still look better than otherwise, so give it a go and see if you like the results.

##### Share on other sites
I had the opportunity to throw in alpha-to-coverage tests. Here are the results. Again, excuse the artifacts at the edges of the leaves; this is a texture clamping error.

Close up, respectively: alpha-blending only, alpha-testing only, blending and testing, two-pass, alpha-to-coverage.

Farther away.

Distant.

You were right, alpha-to-coverage doesn't work at all without anti-aliasing enabled - I must've had AA going when I experimented with it earlier. The above shots were taken with 16x AA on a GF7.

Personally I feel the two-pass method produces the best results. Alpha-to-coverage seemed to be barely better than alpha-testing alone as it didn't look good when moving and the stippled nature of close-ups looks nasty. It's a nice alternative, though, when you have so much foliage you can't afford to render it twice. But if you've got the odd tree or three and small clumps of scrub grass so rendering them twice would be unnoticeable, the two-pass method produces nicer results IMHO.

Do the alpha-to-coverage screen-shots reflect what you're getting, or am I doing something wrong?

##### Share on other sites
Quote:
 Original post by PlayerXYou were right, alpha-to-coverage doesn't work at all without anti-aliasing enabled - I must've had AA going when I experimented with it earlier. The above shots were taken with 16x AA on a GF7.

16x AA on GF7?? Try 4x for comparison since IIRC GF7 can only do 4 "real" MSAA samples (GF8 can do 8).

Quote:
 Original post by PlayerXDo the alpha-to-coverage screen-shots reflect what you're getting, or am I doing something wrong?

Those look remarkably bad compared to what I've seen actually. Unfortunately I don't have an example handy (it was at work when I was trying this all out a year or two ago), but I'll see what I can dig up when I get a chance...

##### Share on other sites
in my gf7's application settings u can choose 8xAA

yes i also thought the alphato coverage shots looked worst than i though, so i fired up one of my apps that does it, and yep it does look a lot worse than i thought. though better than plain alpha testing.
the beauty with alpha to coverage though it works with depthtesting, playerX your method does also but it requires u to draw the (expensive) alpha geometry twice, thus its a lot slower

##### Share on other sites
Quote:
 Original post by zedzin my gf7's application settings u can choose 8xAA

Yes but IIRC 8xS AA (or whatever it's called - something like that) is a hybrid mode that does 2x super-sampling with 4x MSAA.

##### Share on other sites
Quote:
 16x AA on GF7?? Try 4x for comparison since IIRC GF7 can only do 4 "real" MSAA samples (GF8 can do 8).

I tried 4x AA as well. I only showed the 16x because it looked better. GF7 can do 16x AA in an SLI configuration.
Quote:
 the beauty with alpha to coverage though it works with depthtesting, playerX your method does also but it requires u to draw the (expensive) alpha geometry twice, thus its a lot slower

Yep. It's not a win when you have lots of foliage. But if you don't, then it beats the other methods in quality. What the pictures don't show above is how nice it looks when moving. Personally I reckon give the player the option of the method to use. Have a slider in the graphics settings tab for foliage; Speed vs Quality.

I've seen quite a few games that don't have much foliage though, but still use alpha-testing for them. These games could definitely benefit from the two-pass.

##### Share on other sites
Quote:
 Original post by PlayerXI tried 4x AA as well. I only showed the 16x because it looked better. GF7 can do 16x AA in an SLI configuration.

Ahh, SLI, right! I always forget about that... Still looks surprisingly bad for 16 samples (that should give 16 distinct levels of alpha).

Anyways definitely always good to have another method available in the toolbox :)

##### Share on other sites
Quote:
 Original post by AndyTXThose look remarkably bad compared to what I've seen actually.

That's because these shots are a worst-case scenario--black tree with fine detail against a light background.. is guarenteed to look bad with alpha-to-coverage.

In my experience, A2C is far better than this two-pass method, which is terrible for fillrate. Imagine doing a grass field with the two pass method...impossible. How often do you just have one tree standing alone?

Also this is very low-poly-looking tree; most trees in modern applications are higher poly, 5000-8000 at the nearest LOD with a LOT of overdraw for dense foliage. The two-pass method is a clear loser.

##### Share on other sites
Quote:
Original post by AndyTX
Quote:
 Original post by zedzin my gf7's application settings u can choose 8xAA

Yes but IIRC 8xS AA (or whatever it's called - something like that) is a hybrid mode that does 2x super-sampling with 4x MSAA.

does it!!
strange it just saiz 8x in the settings, yould think theyld advertise it as such
after all 2x super-sampling with 4x MSAA is certainly better than 8xMSAA.

no wonder performance always dived when i turned it on

##### Share on other sites
Quote:
 Original post by Matt AufderheideThat's because these shots are a worst-case scenario--black tree with fine detail against a light background.. is guarenteed to look bad with alpha-to-coverage.
Isn't that pretty much the standard scenario for trees? You'll see dark colored foliage against a light colored sky, except at night -- and you can't see jack at night anyway so who cares?

##### Share on other sites
Quote:
 Original post by Matt AufderheideThat's because these shots are a worst-case scenario--black tree with fine detail against a light background.. is guarenteed to look bad with alpha-to-coverage.In my experience, A2C is far better than this two-pass method, which is terrible for fillrate. Imagine doing a grass field with the two pass method...impossible. How often do you just have one tree standing alone? Also this is very low-poly-looking tree; most trees in modern applications are higher poly, 5000-8000 at the nearest LOD with a LOT of overdraw for dense foliage. The two-pass method is a clear loser.

That's seems pretty closed-minded. As AndyTX pointed out, the two-pass is just another tool in the toolbox, there to use when it's most useful. Not every game is set in a dense forest or lush pastures.

No-one said anything about a single tree. But in any case there are scenarios where a single tree might indeed be rendered; say in a garden in an urban area. There are plenty of games set in sparsely forested outdoors areas; most of World of Warcraft, Guild Wars, Half-Life 2, just to name three.

Flat Out, which was the game I discovered the idea being used in, renders hundreds of trees and bushes per frame using the two-pass method and that is not a slow game by any means. For them it was more important to render low-poly trees that looked good whilst moving fast, than it was to have high-poly trees that sparkled with motion.

And talking of worse-case scenarios, you're right, the above pictures are not the best for A2C. I've rendered forests of trees using each method and there were angles where there was no noticeable difference between A2C and two-pass (and some where even alpha-testing looked as good), but there were also other angles where A2C looked worse than even alpha-testing.

The thing was, there wasn't a single angle that made the two-pass look bad. In every instance I tried it matched or exceeded the quality of the other methods, sometimes by a large margin.

For me, two-pass is a clear winner but only for those times I can afford to burn the extra fill-rate (and that's assuming you use the fixed-function pipeline - it may be possible to single-pass it with a custom fragment shader).

But hey, I'm not gonna lose sleep over it if you never use it. :-)

##### Share on other sites
in my experience when rendering foliage, overdraw is by far the biggest problem. Doing two passes will obviously increase this, therefore, when taking into consideration the complex lighting models and large amounts of foliage detail that realism demands, two-pass rendering is a bad idea in the general case.

Alpha-to-coverage is not a perfect solution, yet provides far better performance in a real-world situation.

##### Share on other sites
Im with PlayerX here. If nothing else, this technique works great for screenshots and videos. In worst case, the framerate should be just halved. So, if quality is necessary, I believe its great to have something like that.

Also, with future HW in mind, one could implement this as a "High Quality" setting for trees rendering and leave regular Alpha testing as "Medium Quality" - thus anyone can choose whether he wants sparkling trees or les smooth framerate.

And who says you cant have both worlds ? Since these methods are separate, you can easily switch between them automatically, based on framerate ("Auto detail" setting) or amount of trees in frustum (or near player).

Im definitely going to implement this in my engine (I too am frustrated by regular alpha testing) and am glad PlayerX shared his findings.

##### Share on other sites
Quote:
 Original post by PlayerXFor me, two-pass is a clear winner but only for those times I can afford to burn the extra fill-rate (and that's assuming you use the fixed-function pipeline - it may be possible to single-pass it with a custom fragment shader).But hey, I'm not gonna lose sleep over it if you never use it. :-)

In my eyes, that two-pass technique that you explained looks very nice, and I didn't know about it before. Personally, I'd rather reduce the world detail and do two nice-looking passes than one with artifacts. So thanks a lot for your well-illustrated explanation! [smile]
It's always good to know tricks like that, if I'm doing something similar later it's good to know about it.

• ### Similar Content

• By _OskaR
Hi,
I have an OpenGL application but without possibility to wite own shaders.
I need to perform small VS modification - is possible to do it in an alternative way? Do we have apps or driver modifictions which will catch the shader sent to GPU and override it?
• By xhcao
Does sync be needed to read texture content after access texture image in compute shader?
My simple code is as below,
glUseProgram(program.get());
glBindImageTexture(0, texture[0], 0, GL_FALSE, 3, GL_READ_ONLY, GL_R32UI);
glBindImageTexture(1, texture[1], 0, GL_FALSE, 4, GL_WRITE_ONLY, GL_R32UI);
glDispatchCompute(1, 1, 1);
// Does sync be needed here?
glUseProgram(0);
GL_TEXTURE_CUBE_MAP_POSITIVE_X + face, texture[1], 0);
glReadPixels(0, 0, kWidth, kHeight, GL_RED_INTEGER, GL_UNSIGNED_INT, outputValues);

Compute shader is very simple, imageLoad content from texture[0], and imageStore content to texture[1]. Does need to sync after dispatchCompute?

• My question: is it possible to transform multiple angular velocities so that they can be reinserted as one? My research is below:

• I have this code below in both my vertex and fragment shader, however when I request glGetUniformLocation("Lights[0].diffuse") or "Lights[0].attenuation", it returns -1. It will only give me a valid uniform location if I actually use the diffuse/attenuation variables in the VERTEX shader. Because I use position in the vertex shader, it always returns a valid uniform location. I've read that I can share uniforms across both vertex and fragment, but I'm confused what this is even compiling to if this is the case.

#define NUM_LIGHTS 2
struct Light
{
vec3 position;
vec3 diffuse;
float attenuation;
};
uniform Light Lights[NUM_LIGHTS];

• By pr033r
Hello,
I have a Bachelor project on topic "Implenet 3D Boid's algorithm in OpenGL". All OpenGL issues works fine for me, all rendering etc. But when I started implement the boid's algorithm it was getting worse and worse. I read article (http://natureofcode.com/book/chapter-6-autonomous-agents/) inspirate from another code (here: https://github.com/jyanar/Boids/tree/master/src) but it still doesn't work like in tutorials and videos. For example the main problem: when I apply Cohesion (one of three main laws of boids) it makes some "cycling knot". Second, when some flock touch to another it scary change the coordination or respawn in origin (x: 0, y:0. z:0). Just some streng things.
I followed many tutorials, change a try everything but it isn't so smooth, without lags like in another videos. I really need your help.
My code (optimalizing branch): https://github.com/pr033r/BachelorProject/tree/Optimalizing
Exe file (if you want to look) and models folder (for those who will download the sources):
http://leteckaposta.cz/367190436
Thanks for any help...

• 9
• 14
• 16
• 10
• 17