Fogging used to be so easy. A few years ago, games with very simple shaders could get away with very cheap hardware fog support. Today, many outdoor games can also get away with using the hw fog, or a very simple approach.
An engine with any sort of alpha or multiplicative blending, however, has a greater challenge on its hands when it comes to handling fogging or fading out.
It becomes apparent why this is when you look at the standard fog blending equation :
FinalColor = SceneColor * ( 1 - FogFactor ) + FogColor * ( FogFactor )
This is just a simple lerp between the unfogged scene color and the fog color, based on the fog factor, which varies from 0 to 1 based on the distance to the near & far fog planes.
My fade effect has the same basic form, although with black fog or fading, you can skip the 2nd term.
With multi-pass scenes, including over-style alpha blending, additive blending, etc, the only practical way to support fog or fading looks like so :
Pass 1..N : Compute SceneColor
Pass N + 1 : Pass1 * ( 1 - FogFactor ) + FogColor * FogFactor
First render the whole scene with no fog at all, which accumulates SceneColor into the back buffer, then do a single pass over the scene again, this time using the fog factor & color to blend in the fog.
This second pass can be much faster, because you can use larger batches that the first time through the scene, because you don't need to worry about material types as much.
This morning I added culling world chunks, entities, particles and decals to the fog cylinder. I define the fog extents as an infinite cylinder centered around the player, and can skip drawing objects in the view frustum but outside of this cylinder, since they would be drawn fully fogged anyway.
But, when I made this optimization to reduce draw calls, I got a nice artifact :
The brighter blue patch came due to a mismatch between the culling when drawing the world the first time and the second time during the fog pass. Basically the back buffer is cleared to the fog color, then part of the world was drawn again additively.
At this point, the only things left to fade out are the particles. Right now, they cull to the volume, but the additive particles need to fade to black, and the opaque particles need to fade to the fog color.