Heres after putting the spheres 10 farther away ( z + 10 ): And here is, after resetting spheres Z (as in the first pic), and putting the cam 10 farther from the shapes (cam z - 10):
While writing this post I had a guess, thing is what affect real cam distance is not the cam, but the world window.. Im generating rays from a giving cam position, to a world window that always lays on z = 0, so in reality, I should be moving this frustum(is it really a frustum?) thing alltogether, not just the cam pos... Is that right?
By camera position, you mean the focal point, right? If you are leaving the image plane (world window, or the 'film of the virtual camera') at z=0, and moving the point you are projecting through (the focal point / camera position / 'lens of the virtual camera') what you are really doing is changing the focal length of the virtual camera: You are zooming in and out. This is like holding the back of the camera steady, and extending the lens out.
What you need to do is move the focal point AND the plane together. This will move the camera, and keep the focal length of the camera the same. Moving forward and back will be the easiest case, just add/subtract from z. But you when rotate the camera, you will need to rotate the image plane with it, and then you will need to break out the matrix math.
Alternatively, you can keep the camera position & 'world window' stationary, and just use a matrix to translate and rotate the world around the camera. Similar to how OpenGL has a 'stationary' camera looking down -z, and you rotate the world around it. It's probably the easiest to deal with. There's bazillions of optimized matrix library classes out there that you can use for this.
So yeah, the focal plane should be moving along with the camera, otherwise it doesn't make much sense. Think of the focal plane as "window into the world", you want it to always be in front of your eyes, otherwise if you walk away from it you'll be seeing less and less of the world (as you noticed in your experiments as the result seems to "zoom in"), just as you'll see less of the outside world if you walk away from your window.
“If I understand the standard right it is legal and safe to do this but the resulting value could be anything.”