All right, got it fixed. It's been too long since I last touched graphics stuff so I forgot about the whole 0.5 pixel location thing. Which is one of the reasons that I'm doing this... experiment.
And yes, I realize that pushing to a texture and rendering a quad is the better solution but when I looked for how to do this, all I got was this whole binding a frame buffer to a texture and doing the draw calls to that. Ergo, it seemed pointless to go that route.