I'll explain my process here as best I can without getting too deep into the math. if everything here checks out, I'll look closer at those operations.
steps I take:
1.) create a unit cube (8 vertices) with these bounds:
min/max x: -0.5f to 0.5f
min/max y: -0.5f to 0.5f
min/max z: -0.1f to -1.1f
(if z ranges from positive to negative, the frustum becomes more like an hour glass)
2.) invert my projection matrix
3.) multiply each of my cube's vertices by this inverted projection matrix
I don't know how to apply the view resolution like 800x600 because the thing is already pretty big at this point, with the smallest face of the frustum (the screen) being slightly larger than the actual screen space.
Edited by caibbor, 17 December 2012 - 06:42 PM.