Hi everyone, I'm having a bit of trouble with vertex shaders in DirectX 9. Basically, no matter what I do, my mesh will not show up if I use a vertex shader on it. I'm using D3DXCompileShader to build it from HLSL, then SetVertexShader to use it, and DrawPrimitives to draw the mesh. The mesh draws normally if the shader is NULL, but if I use the output of my HLSL then it doesn't work. At first, I thought it was something wrong with the way matrices are set (I use SetVertexShaderConstantF) but even when I use the following matrix-less shader on a simple square mesh using only 0.0f and 1.0f, it doesn't work:



struct VertexOut
{
float4 Pos : POSITION;
};

VertexOut SHADER_VERTEX(float4 Pos : POSITION)
{
VertexOut Vert;
Vert.Pos = Pos; //set it to the 2D position
Vert.Pos.z = 0.5f; //make sure Z is halfway through the field
Vert.Pos.w = 1; //make sure the homogeneous clip space is right
return Vert;
}

If my understanding of homogenous clip space is right, 0, 0, 0.5 is the center of the screen, though even changing z and w a lot didn't do anything. The mesh I'm using has four vertices-- z is always 0 and x and y are either 0 or 1. I'm completely stumped as to why this won't draw-- I tried versions returning colors and some that don't-- all sorts of things, but no matter what I do, nothing draws on the screen except when I use a NULL shader. Any ideas what could be causing this? Is there a renderstate that needs to be on for shaders to work or something? I had them working ages ago but since then something must have gone wrong, as I haven't touched shaders in forever while working on the logic part of the engine.

EDIT: Apparently my browser decided to post this twice...

For normalized device coordinates (0,0) is the center of the screen, (-1,-1) is the bottom left, and (1,1) is the top right.

Anyway...are you using a pixel shader? What profile are you using to compile your shader? Do you have the debug runtimes enabled? Have you used PIX to inspect the draw call and device state?

Debug output said to use a pixel shader, so I wrote a basic one that always returned blue. When I tested it on an XYZRHW version of the same mesh, it only drew the very top-left pixel, so I increased the 1's to 100's and got a nice blue square in the top left. However, when I applied the vertex shader again, which looks like it should just pass the coordinates directly through as if it was an XYZRHW mesh instead of an XYZ mesh-- it disappeared again. I wasn't aware of PIX, I'm afraid to say-- is there a quick way to use it?

EDIT: well, I grabbed a frame with PIX and I'm still deciphering the information.
EDIT2: Apparently the vertex shader seems to be drawing just fine-- when I look at the Mesh tab on the DrawPrimitive call, the preVS and postVS boxes are identical, though the viewport box shows the square mostly extending off the top-right edge of the screen. Still, even though it looks like the mesh is on the screen, it's still invisible.

EDIT3: Finally! Apparently it was upside-down and the culling was removing it-- now that I know about this amazing tool called PIX I'll try and get the matrices working too. Thanks!

For the matrices make sure that you transpose it before setting it into the shader constants, since shaders expect matrices to be column-major.

Yep, apparently the initial problem was that I wasn't using a pixel shader which is why transposing didn't work initially. Once I started using a pixel shader, it was still untransposed and now that I've transposed them all AND used a pixel shader, now it works. I'm at a loss as to why it worked without a pixel shader several months ago, though...