Hey there.
I came across this little trick a couple of weeks back which has been a god-send for me when rendering alpha-tested foliage. Basically it allows alpha-tested foliage to look as good as alpha-blended foliage without the sorting artifacts.
I still see a lot of commercial and not-so-commercial games using straight ugly alpha-tested foliage that I figure I'm not the only one this could help.
Anyway, ideally we want to draw our trees using alpha-blending so we get nice smooth edges. However, we need to draw the surfaces from farthest to closest or we get artifacts:
So usually we use alpha-testing as we can't really afford the time to sort all those thousands of surfaces we're using to draw our trees with. Here's the same scene using alpha-testing with a cut-off of 0.5:
As you can see the problem with alpha-tested foliage is the hard edges. The foliage tends to shimmer when moving as well. But at least it's simple and fast. This is what most games seem to use.
We can combine both alpha-blending and alpha-testing in the same pass but then we get fringes which can be unsightly:
Now here's the scene rendered using the trick I came across:
As you can see, we get all the benefits of alpha-blended foliage with the simplicity of alpha-tested foliage, and sure looks nicer than either.
Here's the trick (using OpenGL lingo).
First render all your foliage using standard alpha-testing, but with a fairly high alpha cut-off:
glBlendFunc( GL_ONE, GL_ZERO );
glEnable( GL_ALPHA_TEST );
glAlphaFunc( GL_GREATER, 0.75 );
...draw foliage...
Now redraw your foliage using alpha-blending but with a low alpha-test cut-off, depth-writes turned off, and depth testing set to less-than:
glAlphaFunc( GL_GREATER, 8.0 / 255.0 );
glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );
glDepthMask( GL_FALSE );
glDepthFunc( GL_LESS );
...draw foliage again...
glDisable( GL_ALPHA_TEST );
glDepthMask( GL_TRUE );
glDepthFunc( GL_LEQUAL );
As you can see there is the cost of drawing all your foliage twice, but I think the image quality makes it worth it. You don't need to disable depth writing after each tree - draw all your organic foliage using the alpha-testing pass, then do it all again with the alpha-blending pass.
Here's another comparison shot using a pine-tree (alpha-blended, alpha-tested, both):
Static screenshots don't really do it justice. It looks nicer in motion.
Anyway, hope that helps someone as it sure helped me. Apologies if this is a known trick. If that's the case, I figure there are some who don't know it.