Vertex Shader in DX problem

Started by
7 comments, last by Scorp07 16 years, 11 months ago
Hello, I had posted this in the beginners forum, but haven't gotten any possible ideas in a couple days now, so I'm hoping maybe someone that browses here might be able to help. I've been trying to shift gears and move from the fixed function pipeline to shaders, and am not having the best of luck. I'm using the very basic empty .fx file from FX Composer, which follows:

// An empty material, which simply transforms the vertex and sets the color to white.

//------------------------------------
float4x4 matWVP : WorldViewProjection;

//------------------------------------
struct vertexInput {
    float3 Position	: POSITION;
};

struct vertexOutput {
    float4 HPosition : POSITION;
    float4 Diffuse : COLOR0;
};

//------------------------------------
vertexOutput VS_TransformDiffuse(vertexInput IN) 
{
    vertexOutput OUT;
    OUT.HPosition = mul( float4(IN.Position.xyz , 1.0) , matWVP);
    OUT.Diffuse = float4(1.0f, 1.0f, 1.0f, 1.0f);
    return OUT;
}

//-----------------------------------
technique simplest
{
    pass p0 
    {		
		VertexShader = compile vs_1_1 VS_TransformDiffuse();
		
		// Just use the color
		ColorArg1[0] = Diffuse;
		AlphaArg1[0] = Diffuse;
		ColorOp[0] = SelectArg1;
		AlphaOp[1] = SelectArg1;
    }
}




In the DX code, I made a simple class to hold a quad for learning with, which is as follows:

#include "..\include\CVSQuad.h"

void CVSQuad::Render(LPDIRECT3DDEVICE9 pDevice, D3DXMATRIX* vProj)	{

	if (FAILED(pDevice->SetVertexDeclaration( m_pVD )))	{
		SHOWERROR("SetVertexDeclaration failed", __FILE__, __LINE__);
	}

	D3DXHANDLE hTechnique = m_pEffect->GetTechniqueByName("simplest");
	
	if (FAILED(m_pEffect->SetTechnique(hTechnique))) {
		SHOWERROR("Error setting the technique", __FILE__, __LINE__);
		return;
	}
	if (FAILED(m_pEffect->GetTechniqueDesc(hTechnique, &m_TechDesc))) {
		SHOWERROR("Error getting the technique descriptor", __FILE__, __LINE__);
		return;
	}
	if(FAILED(pDevice->SetStreamSource(0, m_VB.GetVertexBuffer(), 0, sizeof(cuCustomVertex::PositionColorTextured)))) {
		SHOWERROR("Failed to set the stream source", __FILE__, __LINE__);
		return;
	}

	if(FAILED(m_pEffect->SetValue("matWVP", vProj, sizeof(D3DXMATRIX)))) {
		SHOWERROR("Failed in setting effect projection", __FILE__, __LINE__);
		return;
	}

    for( UINT p = 0; p < m_TechDesc.Passes; ++p )
    {
            m_pEffect->BeginPass(p);
			pDevice->DrawPrimitive(D3DPT_TRIANGLELIST, 0, 2);
			m_pEffect->EndPass();
    }
}

void CVSQuad::Initialize(LPDIRECT3DDEVICE9 pDevice)	{
	m_Verts = new cuCustomVertex::PositionColorTextured[6];

	m_Verts[0] = cuCustomVertex::PositionColorTextured(-20.0f, 0.0f, -20.0f, D3DCOLOR_ARGB(255, 0, 0, 255), 0.0f, 1.0f);
	m_Verts[1] = cuCustomVertex::PositionColorTextured(-20.0f, 0.0f,  20.0f, D3DCOLOR_ARGB(255, 0, 0, 255), 0.0f, 0.0f);
	m_Verts[2] = cuCustomVertex::PositionColorTextured( 20.0f, 0.0f,  20.0f, D3DCOLOR_ARGB(255, 0, 0, 255), 1.0f, 0.0f);
	m_Verts[3] = cuCustomVertex::PositionColorTextured(-20.0f, 0.0f, -20.0f, D3DCOLOR_ARGB(255, 0, 0, 255), 0.0f, 1.0f);
	m_Verts[4] = cuCustomVertex::PositionColorTextured( 20.0f, 0.0f,  20.0f, D3DCOLOR_ARGB(255, 0, 0, 255), 1.0f, 0.0f);
	m_Verts[5] = cuCustomVertex::PositionColorTextured( 20.0f, 0.0f, -20.0f, D3DCOLOR_ARGB(255, 0, 0, 255), 1.0f, 1.0f);

	m_VB.CreateBuffer(pDevice, 6, D3DFVF_XYZ | D3DFVF_DIFFUSE | D3DFVF_TEX1, sizeof(cuCustomVertex::PositionColorTextured));	
	m_VB.SetData(6, m_Verts, 0);

	ID3DXBuffer *pBuffer = NULL;
	if (FAILED(D3DXCreateEffectFromFile(pDevice, "media/myShader.fx", NULL, NULL, 0, NULL, &m_pEffect, &pBuffer))) {
		SHOWERROR("Create Effect function failed", __FILE__, __LINE__);
	}
	if(pBuffer != NULL)	{
		MessageBox(0, (const char*)pBuffer->GetBufferPointer(), "Effect Compilation Error", 0);
	}

	D3DVERTEXELEMENT9 vElements[] = 	{ 
		{0,  0, D3DDECLTYPE_FLOAT3,   D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_POSITION, 0}, 
		{0, 12, D3DDECLTYPE_D3DCOLOR, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_COLOR,    0},
		{0, 44, D3DDECLTYPE_FLOAT2,   D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_TEXCOORD, 0},
		D3DDECL_END() 
	}; 
	
	if (FAILED(pDevice->CreateVertexDeclaration( vElements, &m_pVD ))) {
		SHOWERROR("CreateVertexDeclaration failed", __FILE__, __LINE__);
	}
}

CVSQuad::CVSQuad()	{
}

void CVSQuad::Release()	{
	m_VB.Release();
	SAFE_DELETE_ARRAY(m_Verts);
}




pDevice is obvious, m_VB is a from a vertex buffer class to encapsulate vertex buffer functions, and I believe everything else is self explanatory. The problem I'm having is this... when I run the app, I get no errors at all, no failed functions and no compilation errors from the shader file, etc. I've even created bugs in the fx file to make sure it was loading, which threw the app, so I know it's being used. But - the quad still gets rendered blue, and t shouldn't. From passing through the shader file, it should be forced to white. A bigger problem is that when I through this simple quad through the FFP, my system runs it perfectly around 3000 fps. When using this version, my fps only ranks around 600 at first and as it runs will slowly continue to drop. To boot, the longer I let the app run, the longer it takes to unload and return me to the IDE. It seems as though there's a memory leak within the render call itself, but I can't figure out where. I'm at a loss as to why this isn't working, and if anyone can throw some info my way, that would be awesome. Thanks much, and if anything else needs to be provided, just shout.. Scorp [edit] I know the error catches are ugly right now... added as many as possible in an attempt to find what the issue is... they'll be cleaned up once I get things running
Advertisement
Might be a bug in a driver. But let's not asume that it is, because 99% it is users bug. I'll trhow some really wild ideas. :)

First of all, I'm not sure because I haven't used FX files, but shouldn't that vertexInput structure contain the color and texture coordinate too ( you are making a declaration that provides those )?

Have you considered that it will *somehow* skip your vertex shader part because you are not passing all the vertex variables? Try changing the position values from the shader, any effect? Also it might be that because you are not passing the color value in the structure it could overwrite your result by the color defined at vertex as default before proceeding to pixel shader. :=)

Have you tried to remove this part from the shader :
ColorArg1[0] = Diffuse;AlphaArg1[0] = Diffuse;


Hope this helped ! :)
Sincerely,Arto RuotsalainenDawn Bringer 3D - Tips & Tricks
Thank you so much for the response, Arex. The good news is that regardless of the outcome, what you pointed out with the missing vertex elements is definitely a problem. I spent so much time looking at Composer's default material that I didn't even notice them removed when I created a basic "empty" material to work with. Then was so wrapped up in my code, I never noticed what was blatently staring me in the face, so thank you for pointing that out for me.

I opened up the file and made the additions thinking that certainly had to be the problem, but even with fixing that, I do get the same results. To answer your questions, I did try changing the positioning around in the FX file, with no changes in the result, so whatever is going wrong is definitely causing it to be skipped, or... something in my DX code is causing it to never actually be used in the first place, even though it's loaded.

{Edit} Oh, and I did also remove the ColorArg assignments as well, but no changes. Just wanted to add that. I've even set the diffuse values to the input positions which creates a nice gradient color shading, and it compiles fine in Composer and works as expected, but again, nothing ever changes with my main app {/Edit}

I'll keep on looking and Googling and experimenting. If any other thoughts happen to come about, I really do appreciate the extra input.

Thanks!
Scorp
You have to call ID3DXEffect::Begin() and ID3DXEffect::End(), you can't just go right to ID3DXEffect::BeginPass() and ID3DXEffect::EndPass(). The loop should look like:
int passCount = effect->Begin();for(int pass = 0; pass < passCount; ++pass){  effect->BeginPass(pass);  // Submit geometry for render.  effect->EndPass();}effect->End();
Thank you so much jpetrie! I swear I should have caught that. It's right there in the documentation, which I've read several times, and still failed to do it right :/

I've added that in, and now I at least have changes happening. Now the quad isn't being rendered at all, or it's not getting positioned where it should be - however this I'm sure is me not having the FX structure setup quite right yet. I can now dig more into that specifically and hopefully make some progress.

Thank you again to both of you - the help is greatly appreciated, as I'm a complete nOOb to the shaders right now (I'm behind the times lol)

Cheers!
Scorp
In working further with this and trying to get my quad rendered, I'm starting to question the view projection matrix that I'm sending the shader before rendering.

Considering that my camera object returns the view and projection matrices in the following code...

...pDevice->SetTransform( D3DTS_VIEW, m_camera.GetViewMatrix() );D3DXMATRIX* viewMatrix = m_camera.GetViewMatrix();D3DXMATRIX*	projMatrix = m_camera.GetProjectionMatrix();D3DXMATRIX	wvpMatrix;D3DXMatrixIdentity(&wvpMatrix);D3DXMatrixMultiply(&wvpMatrix, viewMatrix, projMatrix);pDevice->Clear( 0, 0, D3DCLEAR_TARGET | D3DCLEAR_ZBUFFER, D3DCOLOR_XRGB( 0, 0, 0), 1.0f, 0 ); pDevice->BeginScene();m_MyQuad.Render(pDevice, &wvpMatrix);...


Am I calculating the matrix correctly that the shader needs? I've been going back through the books and documentation on things, and it seems right, but perhaps someone can verify whether it's correct or not?

The matrix is sent to the render method as a D3DMATRIX*, and is then assigned to the shader so it can do its thing...
if(FAILED(m_pEffect->SetValue("matWVP", vProj, sizeof(D3DXMATRIX)))) {		SHOWERROR("Failed in setting effect projection", __FILE__, __LINE__);		return;	}


Thanks a ton for any feedback on this. Trying to eliminate some possibilities here. As a side, I did throw a much larger vertex buffer through the shader, and from watching my FPS it does seem to affecting my fillrate, so I'm loosely assuming that the vertex data is actually getting channeled (which is why I'm looking hard at the positioning transform)

Thanks!
Scorp
SetValue isn't the function you're looking for, I bet. You'll want to use SetMatrix to set the value instead.

The reason there's a difference is that the Effect framework does some extra processing on matrices when setting them to the shader (specifically, it transposes them). The transpose allows the matrix ops to run better.

Hope this helps.
Sirob Yes.» - status: Work-O-Rama.
Actually, I went in and cranked up DX debugging to full tilt, and discovered that my

pDevice->DrawPrimitive(D3DPT_TRIANGLELIST, 0, 2);

is not successful. Since the function only returns D3D_OK on success or D3DERR_INVALIDCALL on fail, it's got me scratching my head as to why it's failing. It's the first time I've had a DrawPrimitive call fail lol.

I'm looking, but haven't come up with the reason yet. From what you mentioned, it has me wondering something about the actual differences in the calls. If only using the SetMatrix and not SetMatrixTranspose or the other specialized matrix assignments, isn't it just setting a matrix, with the size and structure predetermined, whereas when setting a value you're indicating the data and how many bytes of data there is, which would ultimately do the same thing? I'm just curious. From the array of value setting options at hand, it makes me want to believe that one would need to use the more specialized SetMatrix calls, if they have certain transforms already performed... maybe? Not sure, and I'll be reading more about those once I give DrawPrimitive a complete enema =D

Scorp
Just as an update, I did find the problem with the DrawPrimitive failing. I was on the understanding from a statement made in the MS documentation that before you can call the draw methods, the device must have the FVF set or the vertex declaration set. That makes me believe that either one will work. But in fact, even though I had the declaration set, not having the FVF set was causing the failure. I set the FVF and now it doesn't fail.

{Edit} WooT I finally got a triangle! lol I only got it after removing all but the position from my vertices, but at least I'm getting clues now. If the vertices have texture coords, does that mean that *must* have a texture applied for them to show? Because I wasn't loading any textures... I had simply set them up for it because I'll be using them soon obviousy.

I do still have a matrix transform issue as well, but I'll see if I can refresh myself on the proper matrix math to be sending what it should be getting. Right now, as the camera is moved around, the triangles stretch and skew in some cool funky ways. Unfortunately, they're not supposed to lol {/Edit}

Scorp

[Edited by - Scorp07 on May 18, 2007 4:18:19 PM]

This topic is closed to new replies.

Advertisement