The other day the fog shading started showing some strange artifacts. I realized that they were due to overlapping geometry. Whenever you are doing blending, any geometry that intersects other geometry will get blended twice, often leading to double darkening or double brightening.
This was especially bad on the tower level, which had some terrain and tower wall geometry intersecting. The intersecting pixels would blink, getting dimmer and brighter as the camera moved around.
This wouldn't be something that is easy to fix in the level geometry, so I needed a programmatic solution.
I was drawing the opaque scene first, then re-rendering world geometry with a fog shader, blending it in with the blend mode :
srccolor * one + destcolor * ( 1-srcalpha )
Where srccolor already had the fog factor * fog color, and the fog factor was put into alpha as well. Any pixel rendered twice would have the fog factor * fog color added twice, causing the overbright pixels to show up.
After thinking about it in the back of my mind for a few hours, I came up with couple of solutions. One way to fix it would be to use stencil to prevent any pixels from being written twice. I really didn't want to do this, because my game only requires a 16-bit z buffer, and I'd rather save the bandwidth for higher res and/or AA rather than a 24-bit z buffer plus stencil.
A better fix relied on the semi-deferred shading technique. What I do is to render the opaque scene first as normal, then, immediately following all shadows & lighting, copy the back buffer offscreen to a texture.
Then render the opaque world geometry using this texture projectively, applying the fog in a pixel shader, and rendering it opaque, with no blending. This way, if a pixel is rendered more than once, it doesn't matter because the rendering is opaque, so the results don't add up.
Once that was fixed, I also fixed the liquid system to fog properly, then the debris particles, the sparks, bolts and glow shapes. The only remaining non-fogged items are the smoke & mist systems, which I will knock out tonight.
It's been a pain to implement, but I daresay I'm pretty happy with the results. Just a bit of fog or haze goes a long way toward giving the scenes more depth.