The goal is the following:

I need to shoot rays from my camera into the scene every frame. At the intersection points I am creating virtual cameras. Which are then used to render the scene again (or only a bunch of objects). Creating virtual cameras is analogous to creating viewProjection matrices.

The current approach is:

I render the scene from my camera and output the world positions and the normals with multi render targeting to two different textures. So I have two textures on my GPU which contain the world positions and normals of the scene seen from the camera. To create my viewProjections matrices I only need to get the position and the normal from the textures ( I always assume the up-vector to be vec3(0, 1, 0) ). At the moment I am downloading these two textures and create the matrices on the client side, which are then send to the server side again for further rendering.

The models with 200K vertices are by far not absurd. My GPU runs fine at 40-60fps in a scene with about 1 million vertices and goes even higher when not all vertices are in the camera frustum.

Just calculate it at vertex shader for start and then profile to see what is your bottleneck. If it seems that vertex shader calculations are problem then you start to think how to optimize it.

"But wouldn't this mean two texture accesses (normal and position texture) plus matrix construction per vertex thread? "

Yes. But it'll probably end up living in cache and it's saving an entire rendering setup sequence -- it's one less program binding & execution along with all the GPU management overhead that contains.

It's also simpler, so it gets you running and (as kalle_h says) profiling faster.

GPUs are insanely fast -- particularly vertex programmes. It's really quite hard to saturate the vertex side before you're saturating the pixel side.

And you don't need code to run as fast as possible -- you only need fast enough. Fixating on being the besty-best-bestest-best-best instead of "good enough for what you need" is what kills amateur projects.