Greetings,

I'm developing with flash stage3d api, so I can't say for sure what's going on, but it seems that I have antialiasing artifact.

It appears only when I render to texture with any antialiasing. On movement everything much worse - this broken pixels randomly appears on the edge of a sphere. I tried to debug with renderdoc (stage3d api uses dx11 under the hood), but this great tool shows me wrong results. So I'll listen for more experienced guys with pleasure - is it trully problems with antialiased render target or somethinf else? And I'll check my shader more carefully.

Can you post your shader? Is it possible that the values are not clamped and are overflowing?

Also does it do this at all AA levels, i.e. 2 4, 8 and 16?

like bwhiting, I would guess that something is going wrong in your pixel shader. One thing to watch out for with MSAA is extrapolation. When you have attributes from your vertex shader that are passed to the pixel shader (positions, texture coordinates, colors, etc.) they will typically get interpolated to the center point of a pixel. With MSAA you will get edge pixels don't cover the center point but do cover 1 or more sub-sample points inside the pixel. In those cases you will get vertex attributes that are actually interpolated outside of your triangle. To help account for this, GPU's support centroid sampling. Centroid sampling causes attributes to be interpolated to the centroid of all covered sub-sample positions, which prevents extrapolation.

Thank you everyone,

I didn't post my shader because I don't want to scare people - stage3d shaders are assembler like code. I double checked it and it seems correct - I even don't have divisions. And the main thing - without antialiasing I don't have artifacts. So MJP's answer is the explanation I searched for. Unfortunatelly, stage3d doesn't allow to setup sampling or anything like that.

@bwhiiting, for render target AA levels are 0, 1, 2, 3, 4.

@baldurk, I have wrong results in fragment shader. For example in vertex shader I have a varing (vi0) and I can see it's value, but in fragment shader I suddenly have vi1 (not vi0) value filled with zeroes only. So I can't step through the code and check intermediate results (and output color actually is wrong too, although on the screen it's ok). But as you say it can be problem with render target - flash internally draws to some texture and internally draws it to back buffer in the end. And I tried to turn on every option.

Upd.: I checked on different machine and I can't see this artifacts, so I assume that this is my hardware's fault.