Well, don't ignore the vertex shader when you are doing the FXAA pass then.

Well, don't ignore the the vertex shader when you are doing the FXAA pass then.

Ha Ha, you funny

I'm going give this a rest for the day. I'll get back at it tomorrow and hook up a full screen quad. I'll post back then...


Thanks for your help B_

Ok, I've got my code in place for the full screen quad as well as my shader code with all the #defines and #includes. Everything compiles. Now I just need to call the FXAA method but I'm not sure how to call it. The doc says the color must be in sRGB format with luma. How do I calculate that? Also, the FxaaPixelShader method has 16 inputs. Somel of these inputs are for different platforms (ie ps3 xbox pc, console, etc..) What do I pass in for these vars, I'm only targeting PC's? Can you pass in nulls?


ref to specific inputs ->

FxaaFloat2 pos - is this the TU,TV position from my vertex shader?

FxaaTex tex - is this the color in the PS I get from: float4 color = tex2D(TextureSampler, input.TexCoords); or is this the texture it'self ?

Ok, I've got my code in place for the full screen quad as well as my shader code with all the #defines and #includes. Everything compiles. Now I just need to call the FXAA method but I'm not sure how to call it. The doc says the color must be in sRGB format with luma. How do I calculate that? Also, the FxaaPixelShader method has 16 inputs. Somel of these inputs are for different platforms (ie ps3 xbox pc, console, etc..) What do I pass in for these vars, I'm only targeting PC's? Can you pass in nulls?

ref to specific inputs ->

FxaaFloat2 pos - is this the TU,TV position from my vertex shader?

FxaaTex tex - is this the color in the PS I get from: float4 color = tex2D(TextureSampler, input.TexCoords); or is this the texture it'self ?

You can caculate luma like this:


float luma(float3 color)
{
return dot(color, float3(0.299f, 0.587f, 0.114f));
}

pos are the texture coordinates as you say.
Tex depends on the API you are using. If you are targeting SM3 it is the texture sampler I believe.

EDIT:
You said MSAA used to be fine before the screenshot issue. Maybe it would be easier to solve the screenshot issue? Somehow resolving the contents of the multisampled backbuffer into a normal surface and save that?

[quote name='DJTN' timestamp='1315081200' post='4857244']
Ok, I've got my code in place for the full screen quad as well as my shader code with all the #defines and #includes. Everything compiles. Now I just need to call the FXAA method but I'm not sure how to call it. The doc says the color must be in sRGB format with luma. How do I calculate that? Also, the FxaaPixelShader method has 16 inputs. Somel of these inputs are for different platforms (ie ps3 xbox pc, console, etc..) What do I pass in for these vars, I'm only targeting PC's? Can you pass in nulls?

ref to specific inputs ->

FxaaFloat2 pos - is this the TU,TV position from my vertex shader?

FxaaTex tex - is this the color in the PS I get from: float4 color = tex2D(TextureSampler, input.TexCoords); or is this the texture it'self ?

You can caculate luma like this:


float luma(float3 color)
{
return dot(color, float3(0.299f, 0.587f, 0.114f));
}

pos are the texture coordinates as you say.
Tex depends on the API you are using. If you are targeting SM3 it is the texture sampler I believe.

EDIT:
You said MSAA used to be fine before the screenshot issue. Maybe it would be easier to solve the screenshot issue? Somehow resolving the contents of the multisampled backbuffer into a normal surface and save that?
[/quote]


That cannot be done efficiently and some of the hardware here gets temperamental when asked to create a render target or surface that is not the same as the BB’ s Description. In most cases it won’t let you create it at all when MSAA is turned on.

I’ve spent the last 2 weeks trying to implement this #$%! FXAA and it’s impossible. Whoever said all you have to do is drop in the file and pass it your texture cords and your sampler is insane.

I only need AA for the PC platform but I’ve followed the instructions and created the other 16 values to be passed into the method I will be using (PC=1). I’ve had to comment out the code line by line in the h file to find out why the PC=1 method is failing. It is this line at this point:




if(earlyExit)
#if (FXAA_DISCARD == 1)
FxaaDiscard;
#else
return rgbyM;
#endif





If I remove this chunk of code it works but I'm not getting any AA'ing. The "earlyExit" code works off the luma which I'm calculating in the pass prior to the FXAA pass. I'm calculating my Luma in the alpha channel like this:


float4 color = tex2D(TextureSampler, input.TexCoords);

color.rgb = sqrt(color.rgb);

color.a = dot(color.rgb, FxaaFloat3(0.299, 0.587, 0.114));

return color;




My surface format is X8R8G8B8 and my texture samplers are:



sampler2D TextureSampler = sampler_state

{

Texture = <ScreenTexture>;

MinFilter = Linear;

MagFilter = Linear;

MipFilter = Linear;

AddressU = Clamp;

AddressV = Clamp;

};



Anyone have any idea how fix this?

I never got FXAA to work. I scratched the idea because I couldn’t find any support or help on my issues. I then tried to create a render target and apply MSAA. That didn't work either. Multiple render targets and MSAA are just as big of a headache as the FXAA approach was. I'm now researching SSAA but it looks like it's going to be a huge resource hog for barely noticeable anti-aliasing.

I'm dumbfounded that in this day and age of great technology- little has been accomplished in the GPU world for an economical, simple, anti-aliasing solution. What good is a ton of pixels being processed and rendered quickly only to have jagged edges everywhere?

In order to implement FXAA, you have to have a basic understanding of how to program in HLSL, c/c++ and Directx. If you are missing any of these skills, you will have a hard --or impossible-- time implementing FXAA. To be honest, FXAA is very simple to implement in an engine. Please, please dont take this the wrong way, but try running through a few tutorials on how to use shaders and program in direct x, then try the FXAA implementation again.

And remember, FXAA is a post processing effect, which means it is mainly used for a deferred shading engine. If you are using a forward engine, you should use MSAA.

I’m not an idiot and I understand the fundamentals of what is happening in the FXAA shader and HLSL in general. To me it’s convoluted because it’s trying to satisfy every platform there is in an effort to reach a larger audience and make distribution easier for the developer. Not everyone in the world needs an AA solution that handles every platform in one file, increasing the instruction limit unnecessarily. While I’m grateful for the work Tim does, I stand by my interpretation.

It’s comical how you have the time to write me a paragraph judging my skill set and insulting my intelligence yet you provide no input as to the issues I had. I would assume this forum consist of users varying in skills and abilities. I would also assume that the owners of this forum provide this service as a place where people can help one another. Please, please don’t take THIS the wrong way, but your contribution was useless.

I was introduced to FXAA through this forum in a post about jagged edges. Like I’ve said, I’ve forgone the FXAA option and looking for other solutions. MSAA is an option but on multiple render targets, like my current render/post process it quickly becomes sketchy. DirectX9 + MSAA on multiple render targets don’t play well.

I also need to grab several screen frames during my rendering process in this application, which proposes its own limitations with MSAA and the back buffer. Currently I turn on MSAA on the device (if applicable) and then stretchRec the back buffer to a surface, but this is not working as the surface has no AA even though the back buffer does.

Oh well, I’ll figure something out eventually on my own and I won’t come back here for any type of collaborating / contribution until I’m an "expert".

Change all tex2Dlod into tex2D, because tex2Dlod is not supported by SM 2.0. Also use FXAA_QUALITY__PRESET 10.