Sign in to follow this  

Shader PS_3_0 Fails to Run (returns black frame)

This topic is 829 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I finally got a HLSL shader to run with DX9. This code is clean and works. At least with a PS_2_0 shader.

https://github.com/mysteryx93/AviSynthShader/blob/master/VideoPresenter/D3D9RenderImpl.cpp

 

If I instead compile the shader with PS_3_0, however, what I get is a black frame. I mentioned this before but back then the code wasn't working properly. Now the code works.

 

What could be the issue here?

Share this post


Link to post
Share on other sites

I don't know whether this is related, but the pixel format affects the result. If I run the shader on a texture using D3DFMT_A16B16G16R16, I also get a black screen. It works with D3DFMT_A16B16G16R16F, although I still have to get the half-float conversion right (I just tried one library and it failed to compile).

 

D3DFMT_A16B16G16R16F with PS_3_0 returns "something", but that "something" is very different than when running with PS_2_0. I'll see better once the data converts to half-float properly.

Share this post


Link to post
Share on other sites
ps_3_0 shaders always need to be run with a vs_3_0 vertex shader. You can't mix pixel shaders with fixed-function vertex processing, which is what you're doing in your code. If you enable the debug runtimes in the D3D control panel, it will output an error message when you do this. In your case the shader can be very simple: you just need to transform the vertex position by your orthographic matrix, and pass along the other data to the pixel shader.

By the way, you can use D3DXFloat16To32Array from D3DX to convert from half to full precision.

Share this post


Link to post
Share on other sites

I believe D3D9 debug runtimes only worked on Windows XP?
 
I didn't know about D3DXFloat16To32Array and D3DXFloat32To16Array, that will prove useful.

 

Can I find a sample of setting the dummy vertex shader anywhere?

 

Do I simply need to load a dummy vertex shader and call SetVertexShader? Any additional vertex configuration or rendering configuration that needs to be done?

Share this post


Link to post
Share on other sites

I believe D3D9 debug runtimes only worked on Windows XP?


I'm pretty sure that the debug runtime at least worked on Vista and Win7, but I haven't used D3D9 in quite some time so I can't say for sure.
 

Can I find a sample of setting the dummy vertex shader anywhere?


Here's a simple example:
 
float4x4 Projection;

struct VSInput
{
    float3 Position : POSITION;
    float4 Color : COLOR0;
    float2 TexCoord : TEXCOORD0;
};

struct VSOutput
{
    float3 Position : POSITION;
    float4 Color : COLOR0;
    float2 TexCoord : TEXCOORD0;
};

VSOutput MainVS(in VSInput Input)
{
    VSOutput Output;

    Output.Position = mul(float4(Input.Position, 1.0f), Projection);
    Output.TexCoord = Input.TexCoord;
    Output.Color = Input.Color;
    
    return Output;
}
All this shader does is transform the vertex position by a projection matrix, and then pass along the texture coordinates and color to the pixel shader.

Do I simply need to load a dummy vertex shader and call SetVertexShader? Any additional vertex configuration or rendering configuration that needs to be done?


You will also need to make sure that your projection matrix gets set to the appropriate vertex shader constant registers. You do this in the same way that you're setting constants for your pixel shader: get the ID3DXConstantTable for the shader, and then call SetMatrix. Calling IDirect3DDevice9::SetTransform has no effect on vertex shaders, so you no longer need to call that.

One thing I'm not sure about is whether you can use FVF codes with vs_3_0 vertex shaders. FVF codes were a legacy feature for D3D9 that was carried over from D3D8. D3D9 also supports using vertex declarations to specify your vertex buffer layouts, which are a lot more flexible than FVF codes. You can try it with your FVF code first to see if it works, and if you get an error then you can try mapping your FVF code to a vertex declaration. Edited by MJP

Share this post


Link to post
Share on other sites

One person who has done something similar said he's using PS_3_0 with DX9 without a vertex shader without problem.

 

He uses this for PS_3_0, but when I use this line it fails.
m_pDevice->SetFVF(D3DFVF_XYZRHW | D3DFVF_TEX1);

It works when I use this instead, but only for PS_2_0
m_pDevice->SetFVF(D3DFVF_CUSTOMVERTEX)
 

Perhaps the difference between his code and mine resolves around this? I have no clue what this does.

Share this post


Link to post
Share on other sites
You can find the meaning of the various FVF codes here. XYZRHW indicates that the positions are already transformed, and they should be used by the rasterizer directly. I believe it's the same as using D3DDECLUSAGE_POSITIONT in a vertex declaration. They both essentially specify that vertex processing should be disabled. If you want to use that, then you need to pre-transform your vertex positions into screen space, and I believe you also need to make sure that your position has a valid W component.

Share this post


Link to post
Share on other sites

How do I do that?
 
Btw I just found out that D3DFVF_CUSTOMVERTEX was defined in my code.
#define D3DFVF_CUSTOMVERTEX ( D3DFVF_XYZ | D3DFVF_DIFFUSE | D3DFVF_TEX1 )
 
So I suppose it is the vertex declaration that needs to be changed?

HR(m_pDevice->CreateVertexBuffer(sizeof(VERTEX) * 4, D3DUSAGE_DYNAMIC | D3DUSAGE_WRITEONLY, D3DFVF_XYZRHW | D3DFVF_TEX1, D3DPOOL_DEFAULT, &m_pVertexBuffer, NULL));
VERTEX vertexArray[] =
{
  { D3DXVECTOR3(0, 0, 0), D3DCOLOR_ARGB(255, 255, 255, 255), D3DXVECTOR2(0, 0) },  // top left
  { D3DXVECTOR3((float)m_videoWidth, 0, 0), D3DCOLOR_ARGB(255, 255, 255, 255), D3DXVECTOR2(1, 0) },  // top right
  { D3DXVECTOR3((float)m_videoWidth, (float)m_videoHeight, 0), D3DCOLOR_ARGB(255, 255, 255, 255), D3DXVECTOR2(1, 1) },  // bottom right
  { D3DXVECTOR3(0, (float)m_videoHeight, 0), D3DCOLOR_ARGB(255, 255, 255, 255), D3DXVECTOR2(0, 1) },  // bottom left
};

Here's something else I found. The shader code is distorting the image in a weird way. Any idea what could be causing this? Colors are fine.
 
Before.jpg
 
After.jpg
 
 
Also, I'm also not seeing any difference in the output when I comment out these lines, do I need any of this?

m_pDevice->SetRenderState(D3DRS_CULLMODE, D3DCULL_NONE);
m_pDevice->SetRenderState(D3DRS_ZENABLE, D3DZB_FALSE);
m_pDevice->SetRenderState(D3DRS_LIGHTING, FALSE);
m_pDevice->SetRenderState(D3DRS_DITHERENABLE, TRUE);

m_pDevice->SetRenderState(D3DRS_ALPHABLENDENABLE, TRUE);
m_pDevice->SetRenderState(D3DRS_SRCBLEND, D3DBLEND_SRCALPHA);
m_pDevice->SetRenderState(D3DRS_DESTBLEND, D3DBLEND_INVSRCALPHA);

m_pDevice->SetTextureStageState(0, D3DTSS_COLOROP, D3DTOP_SELECTARG1);
m_pDevice->SetTextureStageState(0, D3DTSS_COLORARG1, D3DTA_TEXTURE);
m_pDevice->SetTextureStageState(0, D3DTSS_COLORARG2, D3DTA_SPECULAR);

m_pDevice->SetTextureStageState(0, D3DTSS_ALPHAOP, D3DTOP_MODULATE);
m_pDevice->SetTextureStageState(0, D3DTSS_ALPHAARG1, D3DTA_TEXTURE);
m_pDevice->SetTextureStageState(0, D3DTSS_ALPHAARG2, D3DTA_DIFFUSE);

There's also this code. I can comment out the second part about MatrixIdentity and it still works; is it useful? The first part with MatrixOrthoOffCenter is required otherwise I get no output.

D3DXMATRIX matOrtho;
D3DXMatrixOrthoOffCenterLH(&matOrtho, 0, (float)m_videoWidth, (float)m_videoHeight, 0, 0.0f, 1.0f);
HR(m_pDevice->SetTransform(D3DTS_PROJECTION, &matOrtho));

D3DXMATRIX matIdentity;
D3DXMatrixIdentity(&matIdentity);
HR(m_pDevice->SetTransform(D3DTS_WORLD, &matIdentity));
HR(m_pDevice->SetTransform(D3DTS_VIEW, &matIdentity));
Edited by MysteryX

Share this post


Link to post
Share on other sites

I found the solution to run PS_3_0 shaders. I need to call
m_pDevice->SetFVF(D3DFVF_XYZRHW | D3DFVF_TEX1);
instead of
m_pDevice->SetFVF(D3DFVF_XYZ | D3DFVF_DIFFUSE | D3DFVF_TEX1);

Because I'm removing D3DFVF_DIFFUSE, the vertex buffer structure must be changed from
 

struct VERTEX
{
    D3DXVECTOR3 pos;        // vertex untransformed position
    D3DCOLOR    color;      // diffuse color
    D3DXVECTOR2 texPos;     // texture relative coordinates    
};

 
to
 

struct VERTEX
{
    float x, y, z, rhw; // the transformed(screen space) position for the vertex
    float tu, tv; // texture coordinates
};

Now it works with PS_3_0 shaders!

Share this post


Link to post
Share on other sites

This topic is 829 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this