Jump to content
  • Advertisement
Sign in to follow this  
DJTN

[DX9] Anti-aliasing

This topic is 2630 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

When forward rendering, what AA options do we have other than MSAA?

Here is what I’ve been through over the last month or so and the reasons I couldn’t get them to work.

1. FSAA – tried to implement this as I thought it would be a rather simple solution. Basically you’re just passing in an image and the code is detecting aliased pixels and then blurs them- but I couldn’t ever get it to work. I posted on these forums for help and was told it was for Deferred Rendering only.


2. MSAA – this works great if you render to the back buffer only. I have several render targets and the end user needs to grab screen shots from time to time. I couldn’t get the hardware to allow me to stretchrec or get a graphic stream of an MSAA surface. I either got an error or the texture lost its AA.


3. Edge Detection – I wrote a shader that detects the edges of mesh, basically a depth pass. The edge detection works well but I cannot get the jagged edges to blend or blur with the additional pass. Even with multiple passes. It also has issues when getting closer or further away from objects as well s things get blurred that are not in the depth pass.

4. SSAA - too expensive.

Share this post


Link to post
Share on other sites
Advertisement

1. ...... I posted on these forums for help and was told it was for Deferred Rendering only.


It works well with forward forward rendering.

Share this post


Link to post
Share on other sites

[quote name='DJTN' timestamp='1316987624' post='4865874']
1. ...... I posted on these forums for help and was told it was for Deferred Rendering only.


It works well with forward forward rendering.
[/quote]

Thanks for the reply Esoru.
If I remove the following code it compiles but no AA:



if(earlyExit)
#if (FXAA_DISCARD == 1)
FxaaDiscard;
#else
return rgbyM;
#endif





The "earlyExit" code works off the luma which I'm calculating in the pass prior to the FXAA pass. I'm calculating my Luma in the alpha channel like this:




float4 color = tex2D(TextureSampler, input.TexCoords);

color.rgb = sqrt(color.rgb);

color.a = dot(color.rgb, FxaaFloat3(0.299, 0.587, 0.114));

return color;





I switched to PS 3.0 and set the quality preset to 10 like you said but still no AA

Share this post


Link to post
Share on other sites
The "early Exit" code is important!
Try if(earlyExit) { return rgbyM; }.

For the first run also try FXAA_GREEN_AS_LUMA 1 or remove the sqrt from your luminace calculation.

Share this post


Link to post
Share on other sites

2. MSAA – this works great if you render to the back buffer only. I have several render targets and the end user needs to grab screen shots from time to time. I couldn’t get the hardware to allow me to stretchrec or get a graphic stream of an MSAA surface. I either got an error or the texture lost its AA.


You need to use StretchRect to resolve the MSAA target to a non-MSAA render target. This will filter the subsamples, so that you get an antialiased image.

Share this post


Link to post
Share on other sites

The "early Exit" code is important!
Try if(earlyExit) { return rgbyM; }.

For the first run also try FXAA_GREEN_AS_LUMA 1 or remove the sqrt from your luminace calculation.


Tried all that and it still doesnt compile unless I remove the EarlyExit statement. Moreso, the return piece. If I remove the "return rgbyM" and replace it with "earlyExit = true" it compiles. So I decided to "If /else" the rest of the code so that I'm setting an FxaaFloat4 to return at the end. It compiles but I'm not getting any AA unless I remove my lumma calulation on the prior pass and turn on FXAA_GREEN_AS_LUMA. Then I get some AA- but doesnt using the green channel for luma present some limitations?

At this point I'm just happy to have it working at all...

Share this post


Link to post
Share on other sites

[quote name='DJTN' timestamp='1316987624' post='4865874']
2. MSAA – this works great if you render to the back buffer only. I have several render targets and the end user needs to grab screen shots from time to time. I couldn’t get the hardware to allow me to stretchrec or get a graphic stream of an MSAA surface. I either got an error or the texture lost its AA.


You need to use StretchRect to resolve the MSAA target to a non-MSAA render target. This will filter the subsamples, so that you get an antialiased image.
[/quote]

Thanks MJP - if I cannot get FXAA to work out I'll revisit MSAA. My problem was that I'd lose the AA when I stretchRect to a non-MSAA surface.

Share this post


Link to post
Share on other sites
Ok, I got the FXAA shader to work using luma in the Alpha channel but now the final image is super bright.

I first render all the mesh to a render target / surface with a 32 RGBA format. Then I render that texture on to a another render target / surface with the same format with the following:



float4 color = tex2D(TextureSampler, input.TexCoords);
color.rgb = sqrt(color.rgb);
color.a = dot(color.rgb, FxaaFloat3(0.299, 0.587, 0.114));
return color;



On the last and final pass I use the texture from above to pass into the FXAA shader to render to a full screen quad:




return FxaaPixelShader(input.TexCoords + offsetForDx9, fxFl4, TextureSampler, etc...);





I'm assuming my luma addition code in the prepass is causing the over exposure in the final image.

Share this post


Link to post
Share on other sites
I commented out the SQRT on the output color on the pass prior to FXAA and now the color is not blown out on the final image but I must be calculating my luma wrong. FXAA is working great on curvey objects but say for example, a simple cube - there are lines blinking in and out on the edges when the camera moves. I must be calculating my luma wrong.

I'm not rendering HDR so my color space is in the LDR automatically. My render targets and textures are in ARGB formats. The prepass texture filtering is off and Min, Mag, Mip filters are set to linear for the final pass to FXAA.

Am I missing something?

Share this post


Link to post
Share on other sites
Sounds good,
but remember that you are using 8 bit for luminance value, may be this will be to inaccurate.
For testing purposes try to calulate the luminance on the fly.

#define FxaaLuma(rgba) saturate( dot(rgba.rgb, FxaaFloat3(0.299f, 0.587f, 0.114f) )
BTW: if you return 0 at "early Exit" then you can check how good your threshold/edge detection works.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!