Sign in to follow this  
GraphicsDude

Multisampling: What's really going on

Recommended Posts

I've been reading a lot about Multisampling the past couple of days and there are still a few things that aren't clear that I was hoping someone could clear up. Most of it is just my curiosity of what's really going on during multisampling. First, here's what I do understand. Multisampling is a subset of supersampling. Supersampling (or oversampling) basically renders the scene at a higher resolution then downsamples the result to the desired size. In supersampling the pixel shader is executed for all the samples in the pixel. In multisampling, the shader is only executed once and any texture lookups required by the shader happen only once using the center coordinates of the pixel and the result is shared across all samples. Please correct me if I'm wrong on any of these parts so far. Now to the parts I'm confused on. Most of the documentation I've read all mention something like, "The z/stencil and color values are stored for each sample in the subpixel". 1) What 'color' are they referring to? I thought the shader (i.e. color) was computed once for the entire pixel not stored. 2) The z/stencil sampling really confuses me. Is each sample in the pixel Z depth compared and rejected? Is z sampling outside the polygon allowed? Are all the Z values averaged and used for the final Z which is then depth compared? Is the final depth value averaged over the samples? 3) How is the final pixel color determined. Is it 'the pixel shader output color' * 'percentage of sub-pixels the polygon covers'?

Share this post


Link to post
Share on other sites
First, the shader (or FFP) runs once per pixel. Once this is done, this polygon has Color and Z/Stencil information. You are correct, this is calculated once.

Then, for each multi-sample sample in the render target (i.e. four times with 4x MSAA):
  1. Calculate whether this portion of the pixel (Based on the multi-sample pattern) is covered by the triangle or not. If it's not, continue on to the next sample.
  2. We've determined that the poly is covering this sample, so do the depth/stencil test now (each sample in the render target has its own stored depth/stencil value). If the test fails for this sample, continue on to the next.
  3. We've passed the coverage and depth/stencil tests, so write the color calculated by the shader into the current color sample on the render target.
So, each sample in the render target has separate depth/stencil and color information (Which is why both the depth/stencil and color buffers for a 4xMSAA render target are 4x larger than those for a non-multisampled setup). Note that, while the pixel shader is run based on the pixel center (hence why it's only calculated once), each sample of the MSAA RT has its own position within the pixel at which coverage is evaluated.

Finally, when the render target is resolved (i.e. when displaying to the screen), the color samples are typically averaged to give the final output color that you see on-screen.

Share this post


Link to post
Share on other sites
Thanks! Erik, I've read that article before. It is a good resource.

Drillian, that clears up a lot. Where did you get your information from? I now see the source of my confusion. I thought each pixel was averaged after rendering each polygon, not after rendering all polygons. I've thought about it a few more days and I think I get it now. Please let me know if this is incorrect.

Poly 1:
The pixel shader is ran once. For each pixel the samples are tested for depth/stencil (alpha??) and if the test passes the color value output by the shader is stored in a buffer. This means multiple samples inside the pixel will have the exact same color.

Poly 2:
When another polygon is rendered that shares an edge with the previous polygon above. The same process happens. The shader is ran and each sample in a shared pixel is z/stencil tested and those that pass have the color saved into the buffer.

By this point all the pixels which share an edge between the two polygons have color information stored from BOTH polygons. Just before the buffer is displayed to the screen all the color values samples which are stored per pixel are averaged which give the smooth result.

Share this post


Link to post
Share on other sites
Quote:
Original post by GraphicsDude
Just before the buffer is displayed to the screen all the color values samples which are stored per pixel are averaged which give the smooth result.


Yes, but it doesn't necessarily happen before being displayed to the screen. This was true in the olden days when all games directly rendered to the back buffer, but in modern games it's much more likely that the game will render to an off-screen render target and put that through a series of steps called post-processing. For this type of game you will render to a render target with multisampling, and the resolve (which is where the sub-samples are somehow combined into a single pixel) is performed at some point before the final processed image is rendered to the back-buffer and presented (flipped to the front buffer).

In the DX9 era, the resolve happened as soon as you wanted to read from the multisampled render target. This was because the API didn't support sampling from multisampled textures, so you had to resolve right before post-processing. In DX10 and above it is possible to sample individual subsamples...this gives you more flexibility for certain situations. For instance, if you wanted you could perform a custom resolve instead of just an average. You could also perform HDR tone-mapping for each sub-sample, which produces better results since tone-mapping is a non-linear operation (thus tonemap(subSampleAverage) != avg(tonemap(subsample))). Another key use is for deferred rendering, where you multisample your G-Buffer and then perform lighting for each subsample.

Also keep in mind that newer hardware typically has multisampling modes that does something other than a simple box filter (average) when resolving sub-samples. For instance ATI has "narrow-tent", "wide-tent", and "edge-detect" modes.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this