Here's the setup:
I have a deferred rendering chain that renders all world-space items to several targets, then does screen-space calculations on those targets to get my final result, which is then rendered onto a screen-facing quad in an orthographic projection that is drawn to the backbuffer and presented.
Shorthand:
World content -> World-View-PerspectiveProjection matrix -> gbuffer -> textured to quad -> orthographic projection matrix -> screen.
This is all for a level editor, which lets you paint directly on the terrain and manipulate terrain structure, etc. One thing I'm trying to do is draw the "border" of the paintbrush on the terrain, and I already have reverse projection in place for picking, so I know (in world-space coordinates) where the mouse is pointing at all times. My brush border is a linestrip that is drawn relative to that world-space point.
Here's the specific question:
I want to draw the brush border as it lays on the terrain. I get its normalized coordinates (in the -1,1 range on all three axes) by applying the same worldViewProjection matrix that I use to draw the rest of the world content. I'm trying to draw the resulting linestrip on TOP of the ortho view of the textured quad that shows my deferred results (so that the color of the linestrip isn't affected by any of the pixel shaders that combine my deferred buffers). If both an ortho projection and a perspective projection translate coordinates into the same normalized space, why doesn't the brush circle (technically an ellipse in all but a straight-down view) show up correctly in the view?
A few extra notes:
-My ortho view is built with the viewport width and height of the WinForm it's built into, depth range .1 - 10, with the textured quad at z=10 just to put it as far back as possible.
-The normalized z-depth of the brush radius is usually around +.7 - +.9 given the perspective projection is given a depth range of 4-1000.
-The brush verts are drawn to the backbuffer via a shader that only uses the world-view-projection matrix, AFTER the textured quad is drawn to the backbuffer using the ortho matrix.
-My expectation is that I would get an ellipse (my brush circle from the camera perspective) overlayed on the ortho view of the textured quad. Instead, I get the attached image. This has to be a stupid Linear algebra mistake, but I'm not sharp enough to see it right now.
The green line is my problem. That should be a circle drawn onto the terrain at the mouse point. What am I forgetting or incorrectly assuming?






