So, I've deep dived into my MSAA items, and I'm still thinking about how to properly resolve it?

Previously (and for years) I've just looped over N samples and resolved, but naturally this doesn't need to be done everywhere, because it's just going to waste computation power on samples which don't need to be resolved. Here is a mask of which samples actually have different values (and which have the same):

While it is still quite some, in case of Sponza (Crytek version) we're talking about fraction of pixels requiring resolving. Now - as I'm correctly resolving after all processing in post-tonemap phase, doing all the phases over N samples is going to be computationally intensive (this may count double for calculating GI per-sample). Now - the user can define the number of samples (1, 2, 4, 8) for MSAA.

So, what are my options (throw in more if you can think off any):

1. Do processing per-sample everywhere (the most simple, but wastes a LOT of resources)
2. Store this mask - and then:
   1. For pixels in mask, resolve over ALL samples
   2. For pixels outside of mask, resolve only 1st sample
3. Store this mask - and then:
   1. When processing per-tile (in tile-based approach - but what would be good tile size for this, 32x32 might be too big, 16x16 might be still too big)
      1. If there is any pixel with mask within tile, resolve ALL samples
      2. If there is no pixel with mask within tile, resolve only 1st sample
4. Variant of 3., but apply 2. in case there is any pixel with mask within tile
5. A variant of 2, but execute mask/non-mask pixels separately as 2 separate dispatches? Would require some additional data though (some packing/unpacking of the image) … that could be quite a nightmare to manage

The full pipeline is quite heavy on pixel-level processing (can and will cast cones or rays per-sample/per-pixel).

Now, the problem with 3 is, that some tiles are going to process N times more sample to process. Yet it could actually end up being better than in 2. As in 2, the whole warp (threadgroup) executing the current batch of pixels will wait for the slowest one. Now, this is just a speculation from ray tracers I've worked with in the past - but - could persistent-thread approach help here? That might be a bit too overkill though.

JoeJ said:
Though, everyone seems to agree that MSAA isn't practical with modern standards, and TAA is the only way. Why do you do this? VR support?

This question was something I wanted to avoid - and no it is not primarily VR support. Now, my personal opinion on visual quality (and it's going to be highly subjective) is that every single post-processing AA approach tends to blur and lower the overall quality of image. I do not like the introduced artifacts, problems and/or blurriness visually - and generally I just prefer No-AA at that point (of course speaking about this - with high-DPI 4K displays - No-AA isn't nearly as big problem as it was in the past).

So this being said - I like to give user options (but feasible ones!), allow them to use either MSAA and/or TAA (or other MLAA/FXAA/… variants, but they generally tend to be quite too blurry) or No-AA. I do know there is going to be performance hit for MSAA, the thing is to minimize that hit.

JoeJ said:
Did you consider a compaction step?

I did, I've used something similar for a ray tracer back then in GeForce 7xx generation. It could be quite viable, but I would have to implement and benchmark it. Speaking about this - I could simply store just all samples in large structured buffer and then in resolve map them back to the output pixels with weighting factors.

JoeJ said:
It's the only real solution to the problem. Your other ideas sound more like reducing damage.

That's exactly what I thought. I've tried just looping from 0 to N in a full screen pixel shader - and while that may help in some scenarios, mostly it isn't (as even a single pixel having multiple samples in a warp will reduce the performance of whole warp).

JoeJ said:
Though, a full reduction wouldn't be needed.

That's a fair point - if I split the image into TileSize*TileSize tiles, I can determine whether tile has exactly 1 sample per pixel or more. In case of having just 1 sample per pixel I don't need to do any reduction, just process per-pixel without digging into samples at all. Therefore only tiles that have multiple samples per pixel needs to be processed in a complex way. This being said, I could simply run a reduction of TileSize*TileSize*N samples to build buffer of samples. The advantage of this could be that reduction on smaller scale might be faster… another advantage is, that when user disables MSAA - I just run tiled processing in a single way (no changes).

So… thinking where to begin (now I'm implementing this in toy-app before integrating into main editor and runtime - just to be more flexible). The next step is to switch pipeline to do this (this is more ‘thinking out loud’ part):

1. Render multiple multisampled textures as it is doing now
2. Clear 2 buffers containing tile structure
3. Fill tile buffers in compute shader utilizing the outputs from 1 (first buffer being tiles that have 1 sample per pixel, second buffer being tiles that have at least >1 sample per pixel in tile), this will require atomics when pushing to tile buffers I guess - therefore counting number of tiles in first and second one
4. For each tile in second buffer
   1. Create compact buffer that will hold samples, each with their pixel coordinate/index, at this point I will also know total number of samples in tile
   2. Run processing on samples (I could pre-multiply them here by factor, and therefore during resolve I could just sum the values)
5. For each tile in first buffer
   1. Run processing on pixels (effectively using only 1st sample)
6. Draw first buffer and just display
7. Draw second buffer which will have to include resolve

If I read this correctly, I will need barrier after 2, 3, 4.1 and 5. Edit: Thinking about it, I could just draw indirect tiles for first and second - and that way I won't need to read out counters from 3 at all. I also won't need to read out counters in 4.1. So basically I can collect statistics (which I will probably want to be sure I didn't do any mistake) but I can just read that out at any point, ideally after rendering for frame is finished.

Now the performance hit might be quite high, so with no extensive processing in toy app the difference in timing might be scary. So I may need to add some heavy processing to see whether it paid off or not.

JoeJ said:
Did you try to do a cone trace only for the sample with the highest weight, and using the result for all samples? Probably brings back aliasing, but maybe not that bad?

No, while at the locations with little to no GI (under direct light) you do get antialiasing (although GI contribution is aliased, it is not that visible due to low contribution) - when you look at the ire with mostly indirect contribution (see image), the difference is visible:

I tried to take about the same spot - I still think the voxel resolution also hits the quality (I'm still not happy with it, I've been considering abandoning that for ages - I'm still thinking where to go with that one, I've got about half a dozen (or more) GI implementations in toy projects, and I'm not happy with any single one of those). If you look at the edges in the first image aliasing is clearly visible (which uses 4x for G-Buffer and direct lighting, 1x for VXGI). On the other hand in the second where the edges are nice and smooth it uses 4x for G-Buffer and direct lighting and 4x for VXGI. All resolving is done post-tonemapping step - in both cases. The fps will be lower, as it uses debug build of whole engine (for obvious reason - I just switched startup project to editor to be able to show it in an easy way).

I'm still messing with MSAA approach, got way too busy in past few days (Christmas incoming) to get through it, but I think I'm on a good way. Still no idea how performant it will be.

So, finally had a bit time to take a look at it today. I have a bug in at least one thing (see bottom-most row of tiles, they miss like half of tile for ⅔rd of image - but I can figure that out … Imgui docking/resizing does make a bit of mess in there).

This is how average Sponza will look like with 16x16 tiles - this one doesn't use the sample buffers yet, but do the hard resolve. Each red tile must be resolved, while standard doesn't need to be. With more complex geometry (like lion's head) - there will be no tiles without requirement for sample buffers.

Now it's time to start digging in actual sample buffers.

EDIT: Indeed my resize didn't take place when I docked/undocked window, overall for this purpose it doesn't make a difference. Still at anything with reasonable geometry it requires sample buffers. I'm quite mid-way with them, but might have results soon. Then I'll need to attach GPU profiler and count how much it really takes.

The errors in this one are legendary.

Anyways, I think I've got it. The resolve is a pain though - I think it might be better optimized. The largest part of pain on it are the tiles that need to be resolved. I'll clean up the code and share it here - plus give it another bit of my brain power to analyze how this could be optimized. This is the first time I managed to do it successfully.

Now, in detail - the next image shows:

• Standard colored tiles are resolved by just pass through (no resolve)
• Colored tiles have 2 different pixel statuses in
  • Green are not resolved, just first sample is always used
  • Red are resolved - they require atomic (weighted) addition of multiple samples to resulting value

So, I decided to attach profiler, do a proper release build - and this is the result:

This being said - it can be improved further I believe (many of the edges seem redundant - especially the ones inside the surfaces especially where UV is continuous - like inside the cloth parts, or from the bottom on arches). Reducing the amount of pixels to resolve improves this further.

It can also be further optimized (as ALL tiles are currently reduced - this is the subject on which I'm a bit skeptical - vegetation (I'm doing sample alpha to coverage - which gives me proper transparency “for free” in this case), and complex meshes are going against this.

EDIT: This being said - I believe it still is going to be worth it. The only downside I think about are some post-processing effects that require lookup into neighboring pixels … although I think I might have solution for those.