Hi all,

Here's the setup:
I have a deferred rendering chain that renders all world-space items to several targets, then does screen-space calculations on those targets to get my final result, which is then rendered onto a screen-facing quad in an orthographic projection that is drawn to the backbuffer and presented.

Shorthand:

World content -> World-View-PerspectiveProjection matrix -> gbuffer -> textured to quad -> orthographic projection matrix -> screen.

This is all for a level editor, which lets you paint directly on the terrain and manipulate terrain structure, etc. One thing I'm trying to do is draw the "border" of the paintbrush on the terrain, and I already have reverse projection in place for picking, so I know (in world-space coordinates) where the mouse is pointing at all times. My brush border is a linestrip that is drawn relative to that world-space point.

Here's the specific question:

I want to draw the brush border as it lays on the terrain. I get its normalized coordinates (in the -1,1 range on all three axes) by applying the same worldViewProjection matrix that I use to draw the rest of the world content. I'm trying to draw the resulting linestrip on TOP of the ortho view of the textured quad that shows my deferred results (so that the color of the linestrip isn't affected by any of the pixel shaders that combine my deferred buffers). If both an ortho projection and a perspective projection translate coordinates into the same normalized space, why doesn't the brush circle (technically an ellipse in all but a straight-down view) show up correctly in the view?

A few extra notes:

-My ortho view is built with the viewport width and height of the WinForm it's built into, depth range .1 - 10, with the textured quad at z=10 just to put it as far back as possible.
-The normalized z-depth of the brush radius is usually around +.7 - +.9 given the perspective projection is given a depth range of 4-1000.

-The brush verts are drawn to the backbuffer via a shader that only uses the world-view-projection matrix, AFTER the textured quad is drawn to the backbuffer using the ortho matrix.

-My expectation is that I would get an ellipse (my brush circle from the camera perspective) overlayed on the ortho view of the textured quad. Instead, I get the attached image. This has to be a stupid Linear algebra mistake, but I'm not sharp enough to see it right now.
[attachment=11139:exampleView.jpg]
The green line is my problem. That should be a circle drawn onto the terrain at the mouse point. What am I forgetting or incorrectly assuming?

Is your brush border strip correct? I.e. if you render it as "normal" object in deferred pass, does it appear as ellipse (as it should) or line (as it does now)?
Try hardcoding linestrip coordinates (so it lies on the ground plane) - what happens?

You're on to something here, I finally went in and rearranged my draw code to add the border to the diffuse target, and I get the same result. Here's the code that builds my linestrip:


borderVerts = new VertexPositionColor[16];
lineIndices = new short[17];
double angle = MathUtil.PiOverTwo/4;
for (int i = 0; i < 16; i++)
{
Vector3 temp = new Vector3((float)Math.Sin(angle * i) * radius,
0f,
(float)(Math.Cos(angle * i) * radius));
borderVerts = new VertexPositionColor(temp, new Vector4(0f,1f,0f,1f));
originalBorderVerts = borderVerts; //stores them for an "expand" method if I change radius (not using right now)
lineIndices = (short)(i);
}
lineIndices[16] = 0;



This gave me what I was looking for in XNA, so I'm not sure what's missing...

Followup: this all has to do with how I'm applying a World matrix multiplication, as when I swapped that out with an identity matrix, the circle is showing up perfectly (in either g-buffers or the main backbuffer). So initial question answered: my assumption on normalized spaces is correct. Now I just need to figure out how I'm messing things up with a Translation matrix...

Resolution: always remember when writing your matrix multiplications in the shader code to pass the matrix as the SECOND argument in a mul() call. *sigh*
Thankfully, after a weekend not looking at the problem, 5 minutes reading the code found the obvious mistake.