First HLSL Attempt

Started by
12 comments, last by bababooey 13 years, 5 months ago
Hi,

In finally giving in and switching to HLSL from VS/PS assembly to solve a signed value problem in my simple tweening application, I have created the following effect file which seems to compile fine, but does not end up drawing anything:

//uniform variables set by the app:float4x4 world,view,proj;texture modelTexture;float tweenFactor;float4 diffuseDir;float4 ambientColor;float4 diffuseColor;float4 diffuseAmbientDiff;sampler2D modelTextureSampler=sampler_state{// Texture sampler    Texture=(modelTexture);    MIPFILTER=LINEAR;    MAGFILTER=LINEAR;    MINFILTER=LINEAR;};struct A2V{//Application to vertex shader    float4 pos0:POSITION0;    float4 norm0:NORMAL0;    float2 texCoord0:TEXCOORD0;    float4 pos1:POSITION1;    float4 norm1:NORMAL1;    float2 texCoord1:TEXCOORD1;};struct V2P{//Vertex shader to pixel shader    float4 position:POSITION0;    float2 texCoord:TEXCOORD0;	float4 norm:TEXCOORD1;};void VS(in A2V IN,out V2P OUT){    float4x4 worldViewProj=world*view*proj;    OUT.position=mul(lerp(IN.pos0,IN.pos1,tweenFactor),worldViewProj);//Tween between both positions and project to final coordinates    OUT.norm=mul(lerp(IN.norm0,IN.norm1,tweenFactor),worldViewProj);//Tween between both norms and project to final coordinates    OUT.texCoord=lerp(IN.texCoord0,IN.texCoord1,tweenFactor);//Tween between texture coordinates}float4 PS(in V2P IN):COLOR{    IN.norm=normalize(IN.norm);//perhaps superfluous; needs testing    float NdotL=saturate(dot(IN.norm,diffuseDir));//dot norm with diffuse direction    float4 diffuse=NdotL*diffuseColor*diffuseAmbientDiff;	    return diffuse+ambientColor;}technique Tween{    pass p0{        vertexshader=compile vs_2_0 VS();        pixelshader=compile ps_2_0 PS();    }}


And here are the relevant lines of C++ that call it:
//Init:ID3DXEffect* tweenEffect;if(FAILED(D3DXCreateEffectFromFileW(graphDevice,L"tween.fx",NULL,NULL,NULL,NULL,&tweenEffect,&fehlerMsgs))){	//(Log debug info)}//Then for each model:tweenEffect->SetTechnique("Tween");//used to use device->SetVertexShader() and SetPixelShader()tweenEffect->SetMatrix("proj",&matr);//Used to be SetVertexShaderConstantF()tweenEffect->SetMatrix("view",&matr);//Used to be SetVertexShaderConstantF()tweenEffect->SetMatrix("world",&bMat);//Used to be SetVertexShaderConstantF()tweenEffect->SetTexture("modelTexture",tex[mesh[m][dw].texture]);//was IDirect3DDevice9::SetTexture() beforetweenEffect->SetVector("ambientFarbe",&D3DXVECTOR4(tweenMat.Ambient.r,tweenMat.Ambient.g,tweenMat.Ambient.b,tweenMat.Ambient.a*alpha));//Was SetPixelShaderConstantF()tweenEffect->SetVector("diffuseFarbe",&D3DXVECTOR4(tweenMat.Diffuse.r,tweenMat.Diffuse.g,tweenMat.Diffuse.b,tweenMat.Diffuse.a*alpha));//Was SetPixelShaderConstantF()tweenEffect->SetVector("diffuseAmbientDiff",&D3DXVECTOR4(	tweenMat.Diffuse.r-tweenMat.Ambient.r,	tweenMat.Diffuse.g-tweenMat.Ambient.g,	tweenMat.Diffuse.b-tweenMat.Ambient.b,	tweenMat.Diffuse.a*alpha-tweenMat.Ambient.a*alpha	));//Was SetPixelShaderConstantF()		tweenEffect->SetFloat("tweenFactor",tween);//Was SetVertexShaderConstantF()		tweenEffect->Begin(&passes,0);for(p=0;p<passes;p++){	tweenEffect->BeginPass(p);	graphDevice->SetStreamSource(0,bauartVB[m][dw],0,sizeof(MVERTEX));//hasn't changed	graphDevice->SetStreamSource(1,bauartVB[m2][dw],0,sizeof(MVERTEX));//hasn't changed	graphDevice->SetIndices(bauartIB[m][dw]);//hasn't changed	graphDevice->DrawIndexedPrimitive(D3DPT_TRIANGLELIST,0,0,modelNumV[m][dw],0,modelNumI[m][dw]);//hasn't changed	tweenEffect->EndPass();}


I have directx on debug mode with maximum output, and nothing relevant seems to come in the message window of Visual Studio (then again, nothing complained even when I did have variable names wrong in SetFloat() that didn't match the names in the shader). Parts of the scene not drawn with the shader, ie. fixed-function line lists and text fonts, are still drawn fine. Something glaringly missing? If you'd like me to post the old code that worked, let me know.

I'm on a Dell Studio laptop, i5 proc, ATI Radeon 1GB RAM, DirectX 9.0c SDK (June 2010) on Visual Studio 2010 Express.

Thanks.

--
Bababooey
Advertisement
Have you tried adding proper external variables in your shader?

Ex.
float tweenFactor:TWEENF;

In your C++ code use...
tweenEffect->SetFloat("TWEENF",tween);

Also, I would suggest using handles since it's faster (per MS docs, no string lookup). Also, check your return values which can be done on all set functions as in if(FAILED(yada yada) {
or
HRESULT hr;
hr=tweenEffect->SetFloat("TWEENF",tween);
if(FAILED(hr)) {...

******************************************************************************************
Youtube Channel

I don't really get the whole picture, but I've noticed a strange detail.

tweenEffect->SetMatrix("proj",&matr);//Used to be SetVertexShaderConstantF()tweenEffect->SetMatrix("view",&matr);//Used to be SetVertexShaderConstantF()


I may be missing something, but you seem to pass the same matrix (whatever it holds) as both view and projection and then multiply them in your VS.

Is that what you need?
Quote:
Have you tried adding proper external variables in your shader?

Ex.
float tweenFactor:TWEENF;

In your C++ code use...
tweenEffect->SetFloat("TWEENF",tween);

Also, I would suggest using handles since it's faster (per MS docs, no string lookup). Also, check your return values which can be done on all set functions as in if(FAILED(yada yada) {
or
HRESULT hr;
hr=tweenEffect->SetFloat("TWEENF",tween);
if(FAILED(hr)) {...


Addressing the second part, what's odd is that when I wrap all the Set[Float|Matrix|Technique] calls as they are with if(FAILED(...))exit(-60), none of them fail. Apparently all the names are fine, and still no relevant debug info is output.

As for your first suggestion, what's more odd is that I changed it to use handles in the app, and somehow all the GetVariableByName()'s fail:

D3DXHANDLE shWelt,shProj,shSicht,shTween,shTexture,shAmbientFarbe,shDiffuseFarbe,shDiffuseAmbientDiff,shTechnique,shDiffuseDir;//global and extern'ed in header file//In one-time initialization:if(FAILED(shWorld=tweenEffect->GetParameterByName(0,"world")))MessageBox(NULL,_T("Problem mit GetParameterByName: shWorld."),_T("Fehler"),NULL);return -60;//this fails if(FAILED(shProj=tweenEffect->GetParameterByName(0,"proj")))MessageBox(NULL,_T("Problem mit GetParameterByName: shProj."),_T("Fehler"),NULL);return -61;if(FAILED(shView=tweenEffect->GetParameterByName(0,"view")))MessageBox(NULL,_T("Problem mit GetParameterByName: shView."),_T("Fehler"),NULL);return -62;if(FAILED(shTween=tweenEffect->GetParameterByName(0,"tweenFactor")))MessageBox(NULL,_T("Problem mit GetParameterByName: shTween."),_T("Fehler"),NULL);return -63;if(FAILED(shTexture=tweenEffect->GetParameterByName(0,"texture")))MessageBox(NULL,_T("Problem mit GetParameterByName: shTexture."),_T("Fehler"),NULL);return -64;if(FAILED(shAmbientColor=tweenEffect->GetParameterByName(0,"ambientColor")))MessageBox(NULL,_T("Problem mit GetParameterByName: shAmbientColor."),_T("Fehler"),NULL);return -65;if(FAILED(shDiffuseColor=tweenEffect->GetParameterByName(0,"diffuseColor")))MessageBox(NULL,_T("Problem mit GetParameterByName: shDiffuseColor."),_T("Fehler"),NULL);return -66;if(FAILED(shDiffuseAmbientDiff=tweenEffect->GetParameterByName(0,"diffuseAmbientDiff")))MessageBox(NULL,_T("Problem mit GetParameterByName: shDiffuseAmbientFarbe."),_T("Fehler"),NULL);return -67;if(FAILED(shDiffuseDir=tweenEffect->GetParameterByName(0,"diffuseDir")))MessageBox(NULL,_T("Problem mit GetParameterByName: shDiffuseDir."),_T("Fehler"),NULL);return -68;if(FAILED(shTechnique=tweenEffect->GetTechniqueByName("Tween")))MessageBox(NULL,_T("Problem mit GetTechniqueByName: shTechnique."),_T("Fehler"),NULL);return -69;


This occurs whether or not I prepend the uniform variables with "uniform extern", and whether I add capital-letter arbitrary semantics to the names (using that also in the Get*ByName() calls) as you say. I know I must be noobing it up here hardcore, and this is aside from the main problem of the post. Any idea how to get more info on why it's failing?

Quote:
I may be missing something, but you seem to pass the same matrix (whatever it holds) as both view and projection and then multiply them in your VS.

Is that what you need?

Oh, actually no, I just didn't post the dozen-or so lines of working code between the two that set it up, which haven't changed since it was working with VSA/PSA. These are actually snippets of just the lines that are different since switching to HLSL; they're technically in different functions, and generally executed in the order I posted for each model. If it would help to post my whole code (a few pages) I'd be glad to. Thanks for the observation though.

Speaking of which, I'm still using the same Vertex Declaration and FVF that I was with the working VSA/PSA. Those still apply with HLSL, right?
One thing to be careful about...

ID3DXEffect::SetMatrix will transpose the matrix you give it before calling SetVertexShaderConstantF. This is because (by default) HLSL shaders expect column-major matrices, and the D3DXMATRIX class uses row-major matrices. So you should keep that in mind when porting your code.
Your code always fails because you aren't testing the return value correctly. The FAILED macro expects an HRESULT and you're testing D3DXHANDLE. Part of the problem is your code formatting style. Make free use of braces, tabs, etc., to make it clear at a glance what you're doing.
shWorld=tweenEffect->GetParameterByName(0,"world");if( shWorld==NULL ){   MessageBox(...)   return somevalue;}shProj=tweenEffect->GetParameterByName(0,"proj");if( shProj==NULL ){   MessageBox(...)   return somevalue;}

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

Good eye, Buckeye. How embarrassing that I hadn't surrounded the if-clause body with braces. In correcting that, as well as changing the "FAILED()" to "!()", it now launches without error, as in the OP.

(By the way, when I try LancerSolurus's suggestion of adding arbitrary semantic names to the uniform variable declarations ala "uniform extern float4x4 world:WORLD;" and "tweenEffect->GetParameterByName(0,"WORLD");" it doesn't work, so I just leave it with the plain variable names in GetParameterByName.)

The original problem remains in that it is still not actually drawing anything from the shader, despite no problems from D3DXCreateEffectFromFile(), and no useful debug output is to be found. (And yes I tried removing the D3DXMatrixTranspose() calls before each SetMatrix that I had from when I'd used SetVertexShaderConstantF's, as per MJP's comment about SetMatrix transposing it implicitly.)

Here is the code now, largely unchanged other than for your style and efficiency notes above:

App:
//global and extern'ed:D3DXHANDLE shWelt,shProj,shSicht,shTween,shTexture,shAmbientFarbe,shDiffuseFarbe,shDiffuseAmbientDiff,shTechnique,shDiffuseDir;ID3DXEffect* tweenEffect;//init:if(NULL==(shWorld=tweenEffect->GetParameterByName(0,"world"))){MessageBox(NULL,_T("Problem mit GetParameterByName: shWorld."),_T("Fehler"),NULL);return 60;}if(NULL==(shProj=tweenEffect->GetParameterByName(0,"proj"))){MessageBox(NULL,_T("Problem mit GetParameterByName: shProj."),_T("Fehler"),NULL);return 61;}if(NULL==(shView=tweenEffect->GetParameterByName(0,"view"))){MessageBox(NULL,_T("Problem mit GetParameterByName: shView."),_T("Fehler"),NULL);return 62;}if(NULL==(shTween=tweenEffect->GetParameterByName(0,"tweenFactor"))){MessageBox(NULL,_T("Problem mit GetParameterByName: shTween."),_T("Fehler"),NULL);return 63;}if(NULL==(shTexture=tweenEffect->GetParameterByName(0,"modelTexture"))){MessageBox(NULL,_T("Problem mit GetParameterByName: shTexture."),_T("Fehler"),NULL);return 64;}if(NULL==(shAmbientColor=tweenEffect->GetParameterByName(0,"ambientColor"))){MessageBox(NULL,_T("Problem mit GetParameterByName: shAmbientColor."),_T("Fehler"),NULL);return 65;}if(NULL==(shDiffuseColor=tweenEffect->GetParameterByName(0,"diffuseColor"))){MessageBox(NULL,_T("Problem mit GetParameterByName: shDiffuseColor."),_T("Fehler"),NULL);return 66;}if(NULL==(shDiffuseAmbientDiff=tweenEffect->GetParameterByName(0,"diffuseAmbientDiff"))){MessageBox(NULL,_T("Problem mit GetParameterByName: shDiffuseAmbientDiff."),_T("Fehler"),NULL);return 67;}if(NULL==(shDiffuseDir=tweenEffect->GetParameterByName(0,"diffuseDir"))){MessageBox(NULL,_T("Problem mit GetParameterByName: shDiffuseDir."),_T("Fehler"),NULL);return 68;}if(NULL==(shTechnique=tweenEffect->GetTechniqueByName("Tween"))){MessageBox(NULL,_T("Problem mit GetTechniqueByName: shTechnique."),_T("Fehler"),NULL);return 69;}//per model://(set up worldMat)if(FAILED(tweenEffect->SetMatrix(shProj,&projMat)))exit(-17);//(set up viewMat)if(FAILED(tweenEffect->SetMatrix(shView,&viewMat)))exit(-18);if(FAILED(tweenEffect->SetTechnique(shTechnique)))exit(-19);//TODO: move this to graphRender() or somewhere to be executed only once per frame.graphDevice->SetFVF(D3DFVF_XYZ|D3DFVF_NORMAL|D3DFVF_TEX1);graphDevice->SetVertexDeclaration(graphDecl[0]);if(FAILED(tweenEffect->SetTexture(shTexture,bild[mesh[m][dw].texture])))exit(-10);//(set up material tweenMat)if(FAILED(tweenEffect->SetVector(shAmbientColor,&D3DXVECTOR4(tweenMat.Ambient.r,tweenMat.Ambient.g,tweenMat.Ambient.b,tweenMat.Ambient.a*alpha))))exit(-11);if(FAILED(tweenEffect->SetVector(shDiffuseColor,&D3DXVECTOR4(tweenMat.Diffuse.r,tweenMat.Diffuse.g,tweenMat.Diffuse.b,tweenMat.Diffuse.a*alpha))))exit(-12);if(FAILED(tweenEffect->SetVector(shDiffuseAmbientDiff,&D3DXVECTOR4(	tweenMat.Diffuse.r-tweenMat.Ambient.r,	tweenMat.Diffuse.g-tweenMat.Ambient.g,	tweenMat.Diffuse.b-tweenMat.Ambient.b,	tweenMat.Diffuse.a*alpha-tweenMat.Ambient.a*alpha	))))exit(-13);//(set up world matrix bMat)if(FAILED(tweenEffect->SetMatrix(shWorld,&bMat)))exit(-14);if(FAILED(tweenEffect->SetFloat(shTween,tween)))exit(-15);if(FAILED(tweenEffect->Begin(&passes,0)))exit(-16);for(p=0;p<passes;p++){	tweenEffect->BeginPass(p);	graphDevice->SetStreamSource(0,bauartVB[m][dw],0,sizeof(MVERTEX));	graphDevice->SetStreamSource(1,bauartVB[m2][dw],0,sizeof(MVERTEX));	graphDevice->SetIndices(bauartIB[m][dw]);	graphDevice->DrawIndexedPrimitive(D3DPT_TRIANGLELIST,0,0,modelNumV[m][dw],0,modelNumI[m][dw]);	tweenEffect->EndPass();}if(FAILED(tweenEffect->End()))exit(-16);


Shader:
//unform variables. Adding arbitrary semantics like world:WORLD did not work.uniform extern float4x4 world,view,proj;uniform extern texture modelTexture;uniform extern float tweenFactor;uniform extern float4 diffuseDir;uniform extern float4 ambientColor;uniform extern float4 diffuseColor;uniform extern float4 diffuseAmbientDiff;sampler2D modelTextureSampler=sampler_state{// Texture sampler    Texture=(modelTexture);    MIPFILTER=LINEAR;    MAGFILTER=LINEAR;    MINFILTER=LINEAR;};struct A2V{//Application to vertex shader    float4 pos0:POSITION0;    float4 norm0:NORMAL0;    float2 texCoord0:TEXCOORD0;    float4 pos1:POSITION1;    float4 norm1:NORMAL1;    float2 texCoord1:TEXCOORD1;};struct V2P{//Vertex shader to pixel shader    float4 position:POSITION0;    float2 texCoord:TEXCOORD0;	float4 norm:TEXCOORD1;};void VS(in A2V IN,out V2P OUT){    float4x4 worldViewProj=world*view*proj;    OUT.position=mul(lerp(IN.pos0,IN.pos1,tweenFactor),worldViewProj);//Tween between both positions and project to final coordinates    OUT.norm=mul(lerp(IN.norm0,IN.norm1,tweenFactor),worldViewProj);//Tween between both norms and project to final coordinates    OUT.texCoord=lerp(IN.texCoord0,IN.texCoord1,tweenFactor);//Tween between texture coordinates}float4 PS(in V2P IN):COLOR{    IN.norm=normalize(IN.norm);//perhaps superfluous; we'll see    float NdotL=saturate(dot(IN.norm,diffuseDir));     float4 diffuse=NdotL*diffuseColor*diffuseAmbientDiff;	    return diffuse+ambientColor;}technique Tween{    pass p0{        vertexshader=compile vs_2_0 VS();        pixelshader=compile ps_2_0 PS();    }}


Anything else wrong that would prevent it from drawing? If not, what is the next step in debugging the shader?

Thanks.

--
Bababooey
Your coding style is extremely difficult to read, at least for me. So you may well have other errors similar to some noted previously.

This looks suspicious. The function expects a pointer.
graphDevice->SetVertexDeclaration(graphDecl[0]);

If graphDecl isn't an array of pointers, you probably intend "graphDecl" or "&graphDecl[0]" I suspect.

EDIT:
Quote:Vertex Declaration and FVF that I was with the working VSA/PSA. Those still apply with HLSL, right?

Yes. When using shaders, those declarations "tell" the draw routines how to match up vertex variables with the semantics in the shader.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

Quote:Your coding style is extremely difficult to read, at least for me.

Darn, I had actually taken pains to make it more concise and readable. I find that when code style is bloated with 3 blank lines above and below each statement and spaces between each symbol, it takes me much longer to read and comprehend than with concise chunks at a time. If I expand it out to take up 5 times as much space, would that help you understand it?

Quote:
This looks suspicious. The function expects a pointer.


graphDevice->SetVertexDeclaration(graphDecl[0]);


If graphDecl isn't an array of pointers, you probably intend "graphDecl" or "&graphDecl[0]" I suspect.


That would probably throw a compiler error if it was the case. graphDecl is set up as follows:

#define NUM_VS 2//global:LPDIRECT3DVERTEXDECLARATION9 graphDecl[NUM_VS];//also extern'ed in the header//init:D3DVERTEXELEMENT9 ve[NUM_VS][12]={	{//for tweened models (2 of each to tween between)		{0,0,D3DDECLTYPE_FLOAT3,D3DDECLMETHOD_DEFAULT,D3DDECLUSAGE_POSITION,0},		{0,12,D3DDECLTYPE_FLOAT3,D3DDECLMETHOD_DEFAULT,D3DDECLUSAGE_NORMAL,0},		{0,24,D3DDECLTYPE_FLOAT2,D3DDECLMETHOD_DEFAULT,D3DDECLUSAGE_TEXCOORD,0},		{1,0, D3DDECLTYPE_FLOAT3,D3DDECLMETHOD_DEFAULT,D3DDECLUSAGE_POSITION,1},		{1,12,D3DDECLTYPE_FLOAT3,D3DDECLMETHOD_DEFAULT,D3DDECLUSAGE_NORMAL,1},		{1,24,D3DDECLTYPE_FLOAT2,D3DDECLMETHOD_DEFAULT,D3DDECLUSAGE_TEXCOORD,1},		D3DDECL_END()	},	{//simpler for static models:		{0,0,D3DDECLTYPE_FLOAT3,D3DDECLMETHOD_DEFAULT,D3DDECLUSAGE_POSITION,0},		{0,12,D3DDECLTYPE_FLOAT3,D3DDECLMETHOD_DEFAULT,D3DDECLUSAGE_NORMAL,0},		{0,24,D3DDECLTYPE_FLOAT2,D3DDECLMETHOD_DEFAULT,D3DDECLUSAGE_TEXCOORD,0},		D3DDECL_END()	}};for(i=0;i<NUM_VS;i++)graphDevice->CreateVertexDeclaration(ve,&graphDecl);

All this is unchanged from when it was working using shader assembly.

Thanks for your input though. Any ideas on how to debug the shader itself and find out the difference between what it is doing, and what it's supposed to be?
For debugging shaders, take a look at PIX.
Quote:I had actually taken pains to make it more concise and readable.

[smile]
Perhaps it's just a personal preference, but a few posts ago you found you hadn't used braces.

I'm guessing that if, instead of this:
if(FAILED(shWorld=tweenEffect->GetParameterByName(0,"world")))MessageBox(NULL,_T("Problem mit GetParameterByName: shWorld."),_T("Fehler"),NULL);return -60;//this fails

you had coded this:
if(FAILED(shWorld=tweenEffect->GetParameterByName(0,"world")))   MessageBox(NULL,_T("Problem mit GetParameterByName: shWorld."),_T("Fehler"),NULL);return -60;//this fails

you would have immediately seen the problem. Chances are, you would have seen that you needed braces before you even wrote the line following that. I don't consider adding 3 extra characters (a couple of line returns and a tab) to be "bloat" if it saves a hours of debugging. Something to think about.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

This topic is closed to new replies.

Advertisement