# FXAA Help

This topic is 2312 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

I have the FXAA shader but it's so convolouted. Looked for a tutorial online but nothing turned up.

Anyone know where I can find documentation on FXAA and it's implememtation?

##### Share on other sites
Hrmmm...

The FXAA shader only needs your color buffer as input, but this means you need an extra render target.

So, create an extra render target the same size as your backbuffer.
the FXAA pass should be after all of your game drawing and effects are done.
Copy the color buffer to a temp buffer.
Then, use the copied buffer as input to create the final image.

I am not sure which part you are having a difficult time with.
As for the shader itself, it contains many different implementations for Playstatio, XBox and PC versions. Read the comments, you can safely remove the unneeded code, which will improve your compile times.

##### Share on other sites

I am not sure which part you are having a difficult time with.

The download is a "h" file (header) with tons of macro's (defines). I only need it for the PC. My existing shaders are all FX files and for this specific application (not a game) that requires the AA, it is not written in C++.

Prior to the new requirements, hardware multisampling was sufficient but now they want to take screenshots. I was unable to provide multisampling when rendering to a surface for screen shots- hence, I need an AA alternative.

Will FXAA work with PS v2.0?

Does FXAA access require C++ implementation?

How does FXAA hold up on non- nvidia hardware?

These are the things I'm having a difficult time with...

##### Share on other sites
To add to this, I'm also finding it difficult to create a full screen triangle. How can a triangle be full screen when my backbuffer and monitor are rectangular in shape?

##### Share on other sites
Is it possible to use a sprite and pass that in?

##### Share on other sites

IS FXAA SM 2.0 Compatible?

##### Share on other sites

To add to this, I'm also finding it difficult to create a full screen triangle. How can a triangle be full screen when my backbuffer and monitor are rectangular in shape?

The way to do this is to create a triangle that is big enough so that the rectangle of the backbuffer fits inside. You can also use a full screen quad, if it is easier for you.

IS FXAA SM 2.0 Compatible?

I think it is. Did you put [font="Courier New"]#define FXAA_PC 1[/font] before the include?

##### Share on other sites

Did you put [font="Courier New"]#define FXAA_PC 1[/font] before the include?

Yes, at the top of my shader I put the following:
#define FXAA_PC 1

#include "Fxaa3_11.h"

and it fails to compile. If I only add the include or only the define, then it compiles but with both of them it fails.

##### Share on other sites
Do you have a way to tell why it fails to compile? Looking at the error message of the compiler e.g.

##### Share on other sites

[quote name='smasherprog' timestamp='1314736896' post='4855616']
I am not sure which part you are having a difficult time with.

The download is a "h" file (header) with tons of macro's (defines). I only need it for the PC. My existing shaders are all FX files and for this specific application (not a game) that requires the AA, it is not written in C++.

Prior to the new requirements, hardware multisampling was sufficient but now they want to take screenshots. I was unable to provide multisampling when rendering to a surface for screen shots- hence, I need an AA alternative.

Will FXAA work with PS v2.0?

Does FXAA access require C++ implementation?

How does FXAA hold up on non- nvidia hardware?

These are the things I'm having a difficult time with...

[/quote]

FXAA works great, even on non NV hw. We used it in our commercial project, and this was a concern for us as well. Turns out it was not a problem.
FXAA requires no c++ side implementation, other than a full screen pass. (draw fullscreen quad, set shader, pass in main RT, output to backbuffer or RT of choice).
I don't remember any problems with FXAA on sm2, other than we had to disable the higher quality versions as it resulted in more instructions than supported by sm2. (Memory is a bit hazy though, so don't take my word for it).

##### Share on other sites

Do you have a way to tell why it fails to compile? Looking at the error message of the compiler e.g.

PIX is being buggy as hell and I cannot get it working -however, I did notice that if I switch the order it compiles, I.E.

#include "Fxaa3_11.h"

#define FXAA_PC 1

Once I get pass that problem, how do I call it's method? Looking through to the define I'm working with (FXAA_PC 1) the method ask for all these inputs:

FxaaFloat2 fxaaQualityRcpFrame,
FxaaFloat4 fxaaConsoleRcpFrameOpt,
FxaaFloat4 fxaaConsoleRcpFrameOpt2,
FxaaFloat4 fxaaConsole360RcpFrameOpt2,
FxaaFloat fxaaQualitySubpix,
FxaaFloat fxaaQualityEdgeThreshold,
FxaaFloat fxaaQualityEdgeThresholdMin,
FxaaFloat fxaaConsoleEdgeSharpness,
FxaaFloat fxaaConsoleEdgeThreshold,
FxaaFloat fxaaConsoleEdgeThresholdMin,
FxaaFloat4 fxaaConsole360ConstDir

The documentation says:

Look at the FXAA Quality FxaaPixelShader() for docs on inputs.
As for FXAA 3.11 all inputs for all shaders are the same
to enable easy porting between platforms.

I searched the entire doc for "FXAA Quality FxaaPixelShader()" and never found anything that resembles definitions for these constants / inputs.

SMASHERPROG posted the following earlier in this post saying "The FXAA shader only needs your color buffer as input". Am I missing something here?

##### Share on other sites
I thought I might be hitting an instruc limit so I through together a very simple PS and it still fails to compile:

 #define FXAA_PC 1 #include "Fxaa3_11.h" //texture texture ScreenTexture; sampler2D TextureSampler = sampler_state { Texture = <ScreenTexture>; MinFilter = Linear; MagFilter = Linear; MipFilter = Linear; AddressU = Clamp; AddressV = Clamp; }; float4 PixelShaderFunction(float2 TextureCoordinate : TEXCOORD0) : COLOR0 { float4 color = tex2D(TextureSampler, TextureCoordinate); return color; } technique TransformTexture { pass P0 { PixelShader = compile ps_2_0 PixelShaderFunction(); } } 

I'd like to simplify the Fxaa3_11.h file but not sure what I can and cannot remove since all the methods have inputs to handle every major platform.

##### Share on other sites
Placing the define after the inclusion of the FXAA.h makes no sense though, I believe.
The function you want to call inside your full screen pass is FxaaPixelShader. Try defining [font="'Courier New"]#define FXAA_QUALITY__PRESET 10[/font] and see if it helps, as ATEFred said something about running out of instructions.

##### Share on other sites

Placing the define after the inclusion of the FXAA.h makes no sense though, I believe.
The function you want to call inside your full screen pass is FxaaPixelShader. Try defining [font="Courier New"]#define FXAA_QUALITY__PRESET 10[/font] and see if it helps, as ATEFred said something about running out of instructions.

If I replace #define FXAA_PC 1 with #define FXAA_QUALITY__PRESET 10 it compiles. Should I have added the define or replaced it?

##### Share on other sites
BTW, how are you compiling your shaders? Shouldn't there be a way to see the compile errors?

##### Share on other sites
Ok, I kind of got something going here with my simple pixel shader I posted earlier. I noticed in the Fxaa3_11.h file there are defines for HLSL versions ( #define FXAA_HLSL_3, 4 and 5) but there is no SM 2.0 define. If I switch my PS to:

 #define FXAA_PC 1 #define FXAA_QUALITY__PRESET 10 #define FXAA_HLSL_3 1 #include "Fxaa3_11.h" //texture texture ScreenTexture; sampler2D TextureSampler = sampler_state { Texture = <ScreenTexture>; MinFilter = Linear; MagFilter = Linear; MipFilter = Linear; AddressU = Clamp; AddressV = Clamp; }; float4 PixelShaderFunction(float2 TextureCoordinate : TEXCOORD0) : COLOR0 { float4 color = tex2D(TextureSampler, TextureCoordinate); float value = (color.r + color.g + color.b) / 3; color.r = value; color.g = value; color.b = value; return color; } technique TransformTexture { pass P0 { PixelShader = compile ps_3_0 PixelShaderFunction(); } } 

It compiles but now I'm only getting one color returned and I havn't called any FXAA methods. I need to look at the specs on SM 3 vers SM 2 to see what's going on with that.

I wonder if an older version of FXAA supports SM 2.0? At this time the 3_11 version of FXAA does not. I'm not sure if I can get the requirments lifted to SM 3.0 and how will that impact other areas of this project.

At least I've made "some" progress : - /

Any input would be greatly appreciated....

##### Share on other sites
Oh, I assumed not defining a SM version would default to 2. Seem that I was wrong.
What do you mean by only getting one color returned? Your shader looks like it outputs different shades of gray.

##### Share on other sites

Oh, I assumed not defining a SM version would default to 2. Seem that I was wrong.
What do you mean by only getting one color returned? Your shader looks like it outputs different shades of gray.

Actully it's returning a black and white image. I passed in a simple cloud texture and it when it's Sm 2.0 it returns the texture black and white like it should. When I switch to Sm 3 it returns one color for the entire image.

SM 2.0:

SM 3.0:

##### Share on other sites
Are you getting the same result if you don't include any of the FXAA stuff?

##### Share on other sites

Are you getting the same result if you don't include any of the FXAA stuff?

Yes, I think the issue is that you cannot ignore a vertex shader on a PS 3.0 target. To make sure FXAA wold work with my current rig, I simply rendered everything to a rendertarget/surface and then took that texture and used the Sprite class to draw a full screen sprite with that texture. It works fine for PS 2.0 but not 3.0.

Uggg it's one probelm after another.

##### Share on other sites
Well, don't ignore the vertex shader when you are doing the FXAA pass then.

##### Share on other sites

Well, don't ignore the the vertex shader when you are doing the FXAA pass then.

Ha Ha, you funny

I'm going give this a rest for the day. I'll get back at it tomorrow and hook up a full screen quad. I'll post back then...

##### Share on other sites
Ok, I've got my code in place for the full screen quad as well as my shader code with all the #defines and #includes. Everything compiles. Now I just need to call the FXAA method but I'm not sure how to call it. The doc says the color must be in sRGB format with luma. How do I calculate that? Also, the FxaaPixelShader method has 16 inputs. Somel of these inputs are for different platforms (ie ps3 xbox pc, console, etc..) What do I pass in for these vars, I'm only targeting PC's? Can you pass in nulls?

ref to specific inputs ->

FxaaFloat2 pos - is this the TU,TV position from my vertex shader?

FxaaTex tex - is this the color in the PS I get from: float4 color = tex2D(TextureSampler, input.TexCoords); or is this the texture it'self ?

##### Share on other sites

Ok, I've got my code in place for the full screen quad as well as my shader code with all the #defines and #includes. Everything compiles. Now I just need to call the FXAA method but I'm not sure how to call it. The doc says the color must be in sRGB format with luma. How do I calculate that? Also, the FxaaPixelShader method has 16 inputs. Somel of these inputs are for different platforms (ie ps3 xbox pc, console, etc..) What do I pass in for these vars, I'm only targeting PC's? Can you pass in nulls?

ref to specific inputs ->

FxaaFloat2 pos - is this the TU,TV position from my vertex shader?

FxaaTex tex - is this the color in the PS I get from: float4 color = tex2D(TextureSampler, input.TexCoords); or is this the texture it'self ?

You can caculate luma like this:
 float luma(float3 color) { return dot(color, float3(0.299f, 0.587f, 0.114f)); } 
pos are the texture coordinates as you say.
Tex depends on the API you are using. If you are targeting SM3 it is the texture sampler I believe.

EDIT:
You said MSAA used to be fine before the screenshot issue. Maybe it would be easier to solve the screenshot issue? Somehow resolving the contents of the multisampled backbuffer into a normal surface and save that?

##### Share on other sites

[quote name='DJTN' timestamp='1315081200' post='4857244']
Ok, I've got my code in place for the full screen quad as well as my shader code with all the #defines and #includes. Everything compiles. Now I just need to call the FXAA method but I'm not sure how to call it. The doc says the color must be in sRGB format with luma. How do I calculate that? Also, the FxaaPixelShader method has 16 inputs. Somel of these inputs are for different platforms (ie ps3 xbox pc, console, etc..) What do I pass in for these vars, I'm only targeting PC's? Can you pass in nulls?

ref to specific inputs ->

FxaaFloat2 pos - is this the TU,TV position from my vertex shader?

FxaaTex tex - is this the color in the PS I get from: float4 color = tex2D(TextureSampler, input.TexCoords); or is this the texture it'self ?

You can caculate luma like this:
 float luma(float3 color) { return dot(color, float3(0.299f, 0.587f, 0.114f)); } 
pos are the texture coordinates as you say.
Tex depends on the API you are using. If you are targeting SM3 it is the texture sampler I believe.

EDIT:
You said MSAA used to be fine before the screenshot issue. Maybe it would be easier to solve the screenshot issue? Somehow resolving the contents of the multisampled backbuffer into a normal surface and save that?
[/quote]

That cannot be done efficiently and some of the hardware here gets temperamental when asked to create a render target or surface that is not the same as the BB’ s Description. In most cases it won’t let you create it at all when MSAA is turned on.

I’ve spent the last 2 weeks trying to implement this #\$%! FXAA and it’s impossible. Whoever said all you have to do is drop in the file and pass it your texture cords and your sampler is insane.

I only need AA for the PC platform but I’ve followed the instructions and created the other 16 values to be passed into the method I will be using (PC=1). I’ve had to comment out the code line by line in the h file to find out why the PC=1 method is failing. It is this line at this point:

 if(earlyExit) #if (FXAA_DISCARD == 1) FxaaDiscard; #else return rgbyM; #endif 

If I remove this chunk of code it works but I'm not getting any AA'ing. The "earlyExit" code works off the luma which I'm calculating in the pass prior to the FXAA pass. I'm calculating my Luma in the alpha channel like this:

 float4 color = tex2D(TextureSampler, input.TexCoords); color.rgb = sqrt(color.rgb); color.a = dot(color.rgb, FxaaFloat3(0.299, 0.587, 0.114)); return color; 

My surface format is X8R8G8B8 and my texture samplers are:

 sampler2D TextureSampler = sampler_state { Texture = <ScreenTexture>; MinFilter = Linear; MagFilter = Linear; MipFilter = Linear; AddressU = Clamp; AddressV = Clamp; }; 

Anyone have any idea how fix this?