You are changing the range of the depth valuevec4 position = vec4(texCoord, depth, 1.0); // unpack position.xyz = position.xyz * 2.0 - 1.0;
It's working now. I used the method (poorly) described here and it magically worked: http://bassser.tumblr.com/post/11626074256/reconstructing-position-from-depth-buffer
I'm still struggling a bit with the names of all the coordinate spaces, but I do know that I wasn't passing the correct coordinate space to the inverse projection matrix. I believe it wants NDC space coordinates and I was passing it screen space so I had to divide by w first or something like that, I'm stil looking into it.