Sign in to follow this  

[DX10] Shader Help Needed! ( FIXED! )

This topic is 3199 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi! Okay, I'm starting to think I don't understand a darn thing about shaders. I've been trying for a while to create a general rendering framework for a game, and as part of that I'm studying the PostProcess sample from the DX SDK. ( November 2008, to be exact. ) But since my code is all DX10 based, I'm have some real trouble getting some of the functionality ported over. Specifically, I'm looking at the NormalEdgeDetect post process effect PostProcess SDK sample, and trying to replicate that. I'm rendering to three render targets, just like the DX9 sample, and my intermediate targets look the same as the sample. I've got the "regular" image, a render target with normal information, and a render target with position info. Just like the DX9 sample. All those images match the intermediate output of the sample. BUT...using what seems to be virtually the same shader code as the sample for my postprocess quad render, I get wildly different results. it's clearly SIMILAR to what I want, but something is also way off. I'm posting picture of the sample result, and MY result, to give an idea of what I'm seeing. I'm hoping someone can take a peek and have some idea what is going on. I'm more than willing to post all the code, but it's big, so hopefully someone can give me some idea where to start looking... Anyway, here is what the SDK sample gives, and what I WANT to see... Here is what my version is outputting: And here is the actual shader code, first from the SDK sample:
float4 PostProcessPS( float2 Tex : TEXCOORD0 ) : COLOR0
{
    float4 Orig = tex2D( g_samSrcNormal, Tex );

    float4 Sum = 0;

    for( int i = 0; i < 4; i++ )
        Sum += saturate( 1 - dot( Orig.xyz, tex2D( g_samSrcNormal, Tex + TexelKernel[i] ).xyz ) );

    return Sum;
}




And MY shader code, which SHOULD be doing the same thing...
struct VSNL_OUTPUT
{
    float4 Pos : SV_POSITION;
    float4 Color : COLOR0;
    float2 TexCoords : TEXCOORD0;
};

struct PSNL_OUTPUT{
	float4 color0 : SV_Target;
};

PSNL_OUTPUT PSNL2( VSNL_OUTPUT Input ) : SV_Target
{
	PSNL_OUTPUT output = (PSNL_OUTPUT)0;
	output.color0.a = 1.0f;

    float4 Orig = normalTexture.Sample( samPosition, Input.TexCoords );
     
    float4 Sum = 0;
    for( int i = 0; i < 4; i++ )
        Sum += saturate( 1 - dot( Orig.xyz, normalTexture.Sample( samPosition, Input.TexCoords + TexelKernel[i] ).xyz ) );

    output.color0 = Sum;

    return output;
}




EDIT: you can use regular HTML 'img' tags to embed pictures... [Edited by - dbuzaleski on March 13, 2009 2:26:06 PM]

Share this post


Link to post
Share on other sites
How did you declare the samPosition sampler state? (which I suppose should have been samNormal, but still)
And how do they set up the sampling in the DX9 example? (I'm too lazy to check heheh)

Share this post


Link to post
Share on other sites
"How did you declare the samPosition sampler state? (which I suppose should have been samNormal, but still)
And how do they set up the sampling in the DX9 example? (I'm too lazy to check heheh)"

My samPosition sampler:


SamplerState samPosition
{
Filter = MIN_POINT_MAG_MIP_LINEAR;
AddressU = Clamp;
AddressV = Clamp;
};



The SDK Sample sampler:


sampler2D g_samSrcNormal =
sampler_state
{
Texture = <g_txSrcNormal>;
AddressU = Clamp;
AddressV = Clamp;
MinFilter = Point;
MagFilter = Linear;
MipFilter = Linear;
};



Obviously, there is a difference in that the SDK code binds the texture in the sampler, whereas in my code I'm binding the textures to shader globals and just using the appropriate one as needed. That's also the reason for the naming discrepancy...the SDK samSrcNormal and samSrcPosition do the same thing, they only differ in the texture used, so I'm just using the same sampler in my code for both tasks. ( Hope that made sense instead of confusing the issue more. )

Share this post


Link to post
Share on other sites
Okay...Can't believe it. All my code was correct except for one damn line...

I WAS using...

dstex.Format = DXGI_FORMAT_R8G8B8A8_UNORM;

for my multiple render targets. Instead, I need to use:

dstex.Format = DXGI_FORMAT_R16G16B16A16_FLOAT;

Can anyone explain why that change has such a dramatic effect in this context?

Again, I can post the whole code if anyone needs to see it.

Share this post


Link to post
Share on other sites

This topic is 3199 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this