Original Post
I was going to just post this on my blog (aeolusengine.blogspot), but since I hoped to have some meaningful discussion about it, I decided to post here instead. :) Recently I found myself staring at a random photograph. What stood out was how nice the out of focus areas blurred into discs of light. It reminded me how much I dislike DOF effects in games these days, where it's usually based on gaussian blurs. It looks cheap to me. So I decided to try to implement a good bokeh effect in real-time. The way I looked at it was that the rendered image represents perfectly in focus bundles of light at each pixel. What I wanted to do was take those bundles of light that should be out of focus, and render them as a disc (preferably, though different lenses produce different shapes) over a larger area of the render target (the size of the circle of confusion). So, what are some of the ways you can do this? Rendering ~1280x720 point sprites is out of the question. My first idea was to look at it as rendering the accumulation of n discs at each pixel when rendering a full screen quad. Given a filter: float weights[] = { 0.000000, 0.015703, 0.022473, 0.023882, 0.022473, 0.015703, 0.000000, 0.015703, 0.023882, 0.023978, 0.023978, 0.023978, 0.023882, 0.015703, 0.022473, 0.023978, 0.023978, 0.023978, 0.023978, 0.023978, 0.022473, 0.023882, 0.023978, 0.023978, 0.023978, 0.023978, 0.023978, 0.023882, 0.022473, 0.023978, 0.023978, 0.023978, 0.023978, 0.023978, 0.022473, 0.015703, 0.023882, 0.023978, 0.023978, 0.023978, 0.023882, 0.015703, 0.000000, 0.015703, 0.022473, 0.023882, 0.022473, 0.015703, 0.000000 }; If you take 7x7 (minus the 4 zero weight) samples at a pixel, for each sample look at its circle of confusion radius (stored in the alpha in a previous pass). Then, generate an offset into the filter based on the sample offset and its radius. If the offset puts you outside the filter, we're outside this samples circle of confusion, so it shouldn't contribute to this pixel. Otherwise, add sample * weight[offset] to your result. I got this working. And it looked pretty amazing. It made me see that this effect is well worth the trouble. Unfortunately, there were several problems with this technique. For one, the calculations for building the weight offset were causing the shader compiler to choke with certain optimization settings, and those calculations were in a 7x7 loop (ie very slow). Not to mention taking 7x7 samples per pixel at full resolution is a bit insane. Options? Taking less samples? But then, you get a smaller max circle of confusion radius. I decided instead to generate two fixed disc sizes at 1/4 and 1/8 screen res, and interpolate between these and the unprocessed scene. This removes the need for offset calculations, since the offset is used pretty much directly: for( int x = -3; x <= 3; ++x ) { for( int y = -3; y <= 3; ++y ) { .. weights[x + 3 + (y + 3.f) * 7] .. } } This is much faster. I haven't done gpu timings yet, but for the scene in the screen below, I was getting 100fps. Unfortunately it doesn't look nearly as good as the more accurate method. It's not so much because of the downsized buffers though. 7x7 bokeh at 320x200 actually looks very good if you use filtering when sampling the result. The problem is that blending continuously between two bokeh levels doesn't look very natural, and adds a lot of unwanted blurring and haloing. This is about where I'm stuck. Any ideas? Things I would like: Continuous defocus instead of blending between discrete focus planes. Large circle of confusion radius with as few samples as possible. Random thoughts that might be useful: When storing circle of confusion radius in the alpha channel, you lose the ability to use texture filtering when doing the bokeh passes (since you're basically blurring transformed depth values). If you use texture filtering, you tend to get unexceptable fluttering around depth discontinuities. If you don't use texture filtering, you get a little bit of fluttering elsewhere. D'oh! The artifacts when using texture filtering are much worse though. Would be nice to only filter rgb and point sample a, but that would result in even more sampling. Figuring out how to properly composite the results was a pain. I ended up using MRT to calulate close (in front of focus plane) and far passes simultaneously, but to seperate render targets to ease blending. Basically: sample.a = weights[x + 3 + (y + 3.f) * 7]; output.near += sample * sample_is_near; output.far += sample * sample_is_far; sample_is_near is calculated from the sign of the circle of confusion radius. When calculating the circle of confusion radius, I simple don't abs() the result when shifting the depth value by the focus plane distance. sample_is_far = 1.f - sample_is_near. The alpha (weight) is in sample.a, so you accumulate coverage into the output alpha. This is necessary so you can blend properly when you reject pixels to avoid haloing. When compositing the results: (Continued in next post, ran out of space)...
