Sign in to follow this  
9of9

XNA and Pixel Shader 3.0

Recommended Posts

Hi there! I'm working on a 2.5D XNA game which is quite pixel-shader heavy. So far I've been compiling shaders with PS 2.0, but as I'm experimenting with SSAO at the moment, I found myself hitting the instruction constraints of working with it. But the trouble is, I can't get PS 3.0 shaders to work! They compile just fine and don't get any errors, but they don't seem to have any effect at all. Frustratingly, I haven't found anything on the internet that deals with this issue either, so I'm hoping someone can shed light on what's going wrong. Here is a simple shader which should just render a sprite black:
sampler ColorSampler : register(s0);

float4 ShadowDirPS(float2 texCoord : TEXCOORD0) : COLOR0
{	
	float4 output = float4(0,0,0,0);
	
	output.a = 1;	
	return output;
}

technique Shadowing
{
    pass Pass0
    {
        PixelShader = compile ps_3_0 ShadowDirPS();
    }
}
It works fine if I write 'compile ps_2_0 ShadowDirPS();' at the bottom and I get a black screen, but no effect if I replace it with ps_3_0. This is the code where the shader gets called:
            Game1.spriteBatch.Begin(SpriteBlendMode.AlphaBlend, SpriteSortMode.Immediate, SaveStateMode.None);

            ShadowDir.Begin();

            foreach (EffectPass pass in ShadowDir.CurrentTechnique.Passes)
            {
                pass.Begin();
                Game1.spriteBatch.Draw(DepthPass.GetTexture(), Vector2.Zero, Color.White);
                pass.End();
            }

            Game1.spriteBatch.End();
            ShadowDir.End();
There's probably something really simple and obvious that I'm not doing for it to work, but for all my searching, I've not been able to find anything that would talk about this issue - neither on MSDN nor on any other forums. In case anyone is interested about the project: http://www.youtube.com/watch?v=-Q6ISVaM5Ww

Share this post


Link to post
Share on other sites
Probably a stupid question, but do you have a PS 3.0 compatible video card? :)

Edit: Assuming you're developing for PC, anyway...

Share this post


Link to post
Share on other sites
Quote:
Original post by 9of9
Yes, it's a Radeon 4870 =)


Ah, no clue then! I can only mention that I've run into the same issue with a few shaders in my XNA hobby project and just switching to 3.0 like you describe worked fine. :/

Share this post


Link to post
Share on other sites
Quote:
Original post by lancekt
If he's using SpriteBatch he doesn't need a VS, yeah. (Many of the samples on creators club bear this out.)


Yes he does. Under the hood, XNA's SpriteBatch is using a SM1 vertex shader. You can't mix those with a SM3 pixel shader. You need to use a SM3 vertex shader if you want to use a SM3 pixel shader. Some graphics cards ignore this restriction, but it is the exception, not the rule! You should not do that.

OP will need to go to the SpriteBatch shader page on the Creators Club website, download the stock shader that is used for SpriteBatch and simply edit it to compile under SM3.0. Then he'll need to manually set up and use the shader when he wants to use SM3.0 stuff.

Share this post


Link to post
Share on other sites
Quote:
Original post by Flimflam
Quote:
Original post by lancekt
If he's using SpriteBatch he doesn't need a VS, yeah. (Many of the samples on creators club bear this out.)


Yes he does. Under the hood, XNA's SpriteBatch is using a SM1 vertex shader. You can't mix those with a SM3 pixel shader. You need to use a SM3 vertex shader if you want to use a SM3 pixel shader. Some graphics cards ignore this restriction, but it is the exception, not the rule! You should not do that.


Could you point me to a reference for this? :)

Share this post


Link to post
Share on other sites
Quote:
Original post by lancekt
Could you point me to a reference for this? :)


Well, if you enable debug spew from the debug d3d runtime, it'll tell you right off. In fact, the code won't execute. You'll see this line:
Direct3D9: (ERROR) :ps_3_0 shader can only be used with vs_3_0+ shader (SWVP or HWVP), or transformed vertices.
If anyone is attempting to render SM3 pixel shaders without an appropriate SM3+ vertex shader and the code actually runs, two things are happening:

1) They're not running with the D3D debug runtime (as it enforces this restriction)
2) Their graphics cards are bypassing the restriction and working anyway.

Share this post


Link to post
Share on other sites
Quote:
Original post by Flimflam
Quote:
Original post by lancektCould you point me to a reference for this? :)


Well, if you enable debug spew from the debug d3d runtime, it'll tell you right off. In fact, the code won't execute. You'll see this line:
Direct3D9: (ERROR) :ps_3_0 shader can only be used with vs_3_0+ shader (SWVP or HWVP), or transformed vertices.
If anyone is attempting to render SM3 pixel shaders without an appropriate SM3+ vertex shader and the code actually runs, two things are happening:

1) They're not running with the D3D debug runtime (as it enforces this restriction)
2) Their graphics cards are bypassing the restriction and working anyway.


Ah, thanks! I looked through the MSDN docs (ugh) trying to find some information about this restriction but couldn't. I've been mixing them and hadn't installed the SDK or debug runtime at home--I guess the Nvidia driver is one of the ones that silently allows it, then. Time to switch to the debug runtime!

Sorry for hijacking your thread, 9of9. :)

Share this post


Link to post
Share on other sites
Quote:
Original post by Flimflam
Quote:
Original post by lancekt
If he's using SpriteBatch he doesn't need a VS, yeah. (Many of the samples on creators club bear this out.)


Yes he does. Under the hood, XNA's SpriteBatch is using a SM1 vertex shader. You can't mix those with a SM3 pixel shader. You need to use a SM3 vertex shader if you want to use a SM3 pixel shader. Some graphics cards ignore this restriction, but it is the exception, not the rule! You should not do that.

OP will need to go to the SpriteBatch shader page on the Creators Club website, download the stock shader that is used for SpriteBatch and simply edit it to compile under SM3.0. Then he'll need to manually set up and use the shader when he wants to use SM3.0 stuff.


I see. Thank you, that's very useful and informative. I had a feeling this might be the case, but wasn't sure if SpriteBatch was actually using a vertex shader at all or not, since it does things fairly under-the-hood.

The problem then is, however, how do I set up SpriteBatch to use a different vertex shader for its rendering?

EDIT: Nevermind, didn't spot the readme ^^

EDIT2: Fantastic, it works swimmingly now.

For anyone having this problem in the future, really all you need to do is when writing a SM3.0 shader for use with SpriteBatch, use the stock SpriteBatch .fx file with the vertex shader it provides, while replacing the pixel shader part with the shader you want to use. The only requirement is that you pass your rendertarget resolution as a parameter to the shader at some point.

[Edited by - 9of9 on August 17, 2009 2:15:10 PM]

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this