Direct3D 11 Programming Tip #8: Visualization
This tip is geared towards debugging and understanding the inner workings of programmable shaders. When debugging CPU code, it is pretty easy to set break points, memory watch locations, etc... to figure out exactly what is being done to every variable as you go along. On the GPU things are not quite so nice - it is not simple to get at the step by step computation information for every pixel being drawn.
Direct3D now comes with PIX which goes a long way toward seeing the details of what you are throwing at your API. There are also GPU specific tools that you can use to get some additional information about the functionality of your code (PerfHUD for nVidia, I'm not sure if AMD has something similar), but sometimes they just don't show you what exactly you need to see when you are trying to figure out why your new algorithm isn't working. In addition, back when programmable shaders first showed up there weren't any tools available to help with the task. This meant that you had to roll your own and use your engine to output some useful information. And guess what - even after all these years of tool development, we are right back to having no way of debugging compute shader code (PIX currently doesn't support CS!) so it pays to be able to do your own debugging with your engine.
Debugging and Visualization
So what does debugging have to do with visualization? With the GPU, your primary output path is a render target. This is somewhat diluted in D3D11 since the compute shader can write to just about any type of resource, but still the primary output is the render target. There is also a stream output capability, but that is only available at the geometry shader so we will consider its use for debugging limited.
This means the quickest way to see what you are getting out of your algorithm/shader code is putting it out on a render target and displaying it on screen. But isn't that what we normally do with rendering code - put things onto the screen? The fact is that we do visualizations of our algorithm every time we run a shader. For example, I've been working my ass off on a SSAO implementation. Late last night, I generated the following sequence of images:
Even though I didn't mean to, I was outputting useful information into my render targets that I could use to figure out what was the problem. From those images, you can see that only in a small radius around each object there is a darkened halo which is common for SSAO implementations that don't account for objects in the foreground occluding objects in the background. The remainder of the image shows that the scalar occlusion value for any area that isn't strictly occluded is either jumping wildly or becoming invalid (INF, or NAN). Despite the interesting look of the images, it told me to look at the scalar occlusion calculation and figure out why for relatively flat areas it was generating such harshly varying output.
Scalar and Vector Visualizations
I mentioned 'scalar' visualization in that last example. This is a distinction that needs to be made - you can visualize both scalar (a single component value) and vector (multiple components) attributes. The normal rendering process utilizes scalar color values, with one scalar value representing red, green, and blue. It is often possible to do some type of color mapping to represent some scalar value from your shader code on screen. For example, if a particular scalar value represents the result of a power instruction then the result is likely to be either very large or very small. So you can define a scaling for that value to map it into a [0,1] range for output into one or more of the color channels.
Those same three color channels can also be used to visualize vector values too. In both of these cases, we need to be concerned about the range of the values that will be visualized in the output render target. The classical example of this is trying to visualize the normal vector of a given scene. Since the components of a normal vector typically vary in the range (-1,1), we need to re-scale the value so that it can fit into a (0,1) range. The components are usually scaled and biased (N * 0.5 + 0.5) before being stored into the render targets. This results in each component appearing as 0 when point along it's negative axis, and as 1 when pointing along its positive axis. Here is a sample image showing this type of visualization:
With both scalar and vector visualizations, you can start to put together some quick tests for developing shaders. If something doesn't seem like it is performing the operation that you think it should be, then simply output that value to one of your color channels and scale it accordingly to fit in that channel. You can very quickly identify what is working and what isn't, and work backward to figure out what is happening from there. This also fits nicely with the pre- and post-conditions of a set of shader library functions (if you define them!). You can test to see if your pre-conditions have been violated or not, and quickly eliminate large portions of your shader while debugging.
Visualization is actually quite a large topic in itself. If you are interested in reading more about how scientific visualization is done and what types of operations are usually used to allow very useful ways to look at data, then I would suggest taking a look at the Visualization ToolKit (VTK) website. The VTK is an open source C++ library with bindings for several popular scripting languages (including Python and Java), and provides a data flow style visualization system. It's typically used for pretty advanced visualization topics, but since it is open source you can find some good reference implementations of particular algorithms, and there are tons of examples in the framework as well.
