For reference purposes, a more up-to-date version is now available as part of the DirectX 9 SDK: HDRPipeline
[caution] Warning: This is a fairly long announcement/article type thread, I hope you like it and remember - I really appreciate any comments/feedback!
Introduction
This thread is about a piece of research that was originally for my own personal amusement, but part way through I decided that it might well be of interest/use to you guys. So,
here it is.. [smile]
Whilst HDR rendering is not the most complicated process to work with, it can be quite confusing. As shown by this sample there are a large number of connected steps that go into the final image - each of these steps is straight-forward, but it is the system as a whole that can be confusing.
What I planned to do with this sample that I hadn't seen in any of the other samples I've looked at was show all of these individual steps as part of the GUI. All of the elements are updated each frame, so it should make it easier to understand the system as a whole if you can see what is actually stored/computed in each step.
There are four main sections:
- Rendering the original HDR image. This is the "traditional" part of the pipeline, with the difference that we can use images outside the 0.0 to 1.0 range (or 0..255 range).
- Computing the average luminance for the scene. The result of this is a value indicating how bright (or how dark) the scene is.
- Performing any post-processing on the image. The lens on a camera (as well as the human eye) can "scatter" some of the incoming light and lead to various blurring effects around particularly bright objects.
- Compositing these steps together to create the final result. Taking the results from the previous 3 sections we can create a final image, mapped back down to a regular 32bit (or even 16bit) XRGB texture that is suitable for being displayed to the end user.
Overview
To interpret the GUI correctly, following the arrows in the red circles. Starting from the HDR image in the middle-left, through to the large image (bottom-right).
(Click To Enlarge)
In the above screenshot the application is displaying a regular image that is contained entirely within the 0.0 to 1.0 range (
note that there the 4 cells at the top are all black).
(Click To Enlarge)
This image shows the view having been rotated around (
Drag with RMB). The three faces of the cube are being rendered with colours
above 1.0 - that is, they are particularly bright.
Because the average luminance (
Explained further down..) is much brighter the final image is adjusted such that it is much darker. This is best observed by looking at the background around the cube - the surrounding geometry and the blue background are darker than in other images.
The bright pass at the top is now "active", and is simulating the way that the bright (over exposed) parts of the image are being blurred by the lens.
(Click To Enlarge)
The above screenshot shows a more typical scene - where you'd have a combination of "over bright" and "normal" colours. Due to the luminance calculations, the bright areas of the scene lead to a darkening of the image, but not as much as in the previous image. Because there is still a substantial part of the scene that
isn't bright it draws the luminance average down, leading to the bright areas being over-exposed (and appearing nearly white).
Luminance Path
The luminance calculations are shown in the bottom-left part of the GUI:
Luminance is computed in a GPU-friendly manner - by using multiple (down sampled) render targets and pixel shaders the work can be offloaded to the GPU rather than use the CPU. If we were to measure luminance using the CPU it would require a lot of data to be transferred back across the AGP/PCI-E bus, which is
A Bad Thing™.
Post-Processing Path
The post processing is displayed via the four cells along the top of the GUI:
As previously mentioned, the lens used to capture an image often has particular optical properties - one of which is that it can,
even very slightly, scatter/reflect/refract the incoming light. The brighter this light is, the more pronounced the effect will be. Depending on the lens you can get different types of effects - a regular "bloom" (as in this sample) or a variety of star/cross filters.
The first part of post-processing is picking the pixels that should contribute. To be physically accurate, all should be processed, but for a bit of "artistic licence" most implementations only consider pixels above a certain threshold. There is a slider in the main GUI that allows you to change this to see what effect it has - around 0.5-0.6 works well.
The actual post-processing can be quite a performance-heavy piece of code (lots of texture reading), so the original image is subjected to two down-sampling steps. This reduces the number of pixels that are processed by 64x.
To achieve a 2D blur effect, it can be quite efficiently approximated using two passes - one for a horizontal blur and one for a vertical blur. The combination gives the desired 2D effect. These two passes are seen as the final two steps in the above image.
Final Image Composition
The final result can be seen as the largest image in the GUI:
In a real/production HDR pipeline, the above image is all that you would show to the end-user. All of the other intermediary steps shown in this sample would be kept "behind the scenes".
The pixel shader that outputs this particular image, takes (as shown by the arrows in the GUI) the original HDR image, the final post-processing result and the luminance calculation as inputs. The key part is the luminance input - the value read from this texture is used to "luminance map" the original HDR image and the post-processing results. The net result is that if the scene is deemed to be fairly bright, all the colours are scaled down; if the scene is fairly dark, the colours are scaled up.
Using The Sample
You can download the source code and required files
here (505kb).
Unzip the archive to a folder on your machine and refer to the 'readme.txt' included - it has all of the important details.
An obvious requirement is to have hardware that supports 'Shader Model 2.0' and floating-point textures (either 64bit or 128bit). Also, I developed the code using the August 2005 DirectX SDK - so some people may well require the d3dx9_27.dll to run the sample. In this case I suggest you have a look
here or
here [smile]
References
What I've presented in this sample/thread is far from ground-breakingly original work. In truth, it is just my adaptation and interpretation of a lot of other peoples work.
Hopefully it'll be useful to people though
I can't find links to every single source, but the following list is my recommended further reading:
Articles
Real-Time Glow
High Dynamic Range Rendering
High Dynamic Range Environment Mapping On Mainstream Graphics Hardware
Samples/Demos
HDRLighting
HDRFormats
RTHDRIBL (Real-Time High Dynamic Range Image-Based Lighting)
[Edited by - jollyjeffers on March 2, 2006 6:56:55 AM]
