I've managed to implement screen-space reflections into my forward renderer and now I've been trying to go the step further by rendering recursive reflections (i.e. reflections of reflections of reflections).

I've been trying to do this in a single-pass, by:

1. rendering the camera view of a scene to a 2d texture in the first frame, then
2. feeding this texture back into the same shader pass in the next frame, performing the SSR calculations in this shader pass using this texture.
3. then adding the results to the original color results -> this is then rendered into the same 2d texture as Step 1 and the feedback loop continues.

Here are my results:

[attachment=17892:giboxssr3.png]

Several issues are immediately noticeable:

The most noticeable one on the screenshot above is that reflections within reflections become distorted.

I suspect that this is due to the normals used in the SSR calculations, which continually use the original normals of the scene, thus the higher order reflections are incorrectly using the normals of the base geometry (eg. look at the reflections on the floor).

I believe that a solution to this may be to reflect the normals as well and store this into a new texture - which would be used in every subsequent normal calculation of the SSR algorithm.

Another big problem, which you won't be able to notice in the screenshot above, is any movement that occurs in the scene will show a lag between the original color and the reflected color. This only happens when I feed the texture of the scene back into the scene in the same render pass. If I use two passes, then this doesn't occur, but then I won't get the advantage of the free recursive reflections. I guess this is because, using a single pass, the lag is because the renderer has to wait until all the other effect calculations, such as shadows, global illumination, etc, are completed.

I've also tried putting aside the recursive attempt by using a single reflection so that I can address some of the other issues.

The main issue is how to mask what the camera cannot see - at the moment these areas are black in the reflections because there is no information available in the captured scene from screen space because they are either occluded by other objects, or are outside of the view.

[attachment=17895:giboxssr4.png][attachment=17896:giboxssr5.png]

For the areas that are outside of the view, even if I use fade, they are still noticeable.

Are you using the previous frames view matrices when you unproject the previous frame's data? This might be cause of your sliding artifact if you are using the current one. You can warp the old image by unprojecting it using the old matrix into world space and then reprojecting it using the current matrix.

Also, as Vilem Otte said, you probably don't want to do this at full resolution. This actually might help you since you can also generate the geometry normals as you downsize the depth using the 4 depth values at each pixel. This might give you less artifacts than using the bumped normals. I'm not sure though I haven't tried it. I might too faceted.

For unknown areas you could mark them some way like with the alpha channel in the clear color and then smudge out known pixels into unknown pixels a few steps a frame. If you clear to 0 alpha and every frame dim the alpha a little with valid pixels writing alpha 1 then you can smudge in based on the age of that pixel. We call this algorithm "texture bleeding" at work but Google says that's not the common name. But it's simple.

For unknown areas you could mark them some way like with the alpha channel in the clear color and then smudge out known pixels into unknown pixels a few steps a frame. If you clear to 0 alpha and every frame dim the alpha a little with valid pixels writing alpha 1 then you can smudge in based on the age of that pixel. We call this algorithm "texture bleeding" at work but Google says that's not the common name. But it's simple.

I'm not quite sure if I understand what you are saying. Do you mean for the unknown pixels, I should store a value for them in the alpha channel, then in the next frame, I would sample the value of the previous frame at that screen space location - effectively smudging the area?

Have you got an example that you can show me? Or do you know of anywhere that this technique has been applied before?

Yeah basically. A pixel shader that does it looks like this

Diverging a bit more off topic, in response to Vilem Otte:

Great stuff. I was considering prototyping nearly the exact same thing early this year, but I've been pulled away with other things to do and I still haven't gotten around to it. My reply here will mostly be in regards to your remark on BVHs and optimization (essentially, if you want to be general, the "visibility function"). I also predicted that visibility determination will be the most critical point to implementing this technique. Since I've worked on level modelling tools, I'm familiar with processing geometry. I realized it will be more effective to avoid any secondary scene graphing techniques and consider the content to be a scene graph itself. Back face culling is capable of assumatively omitting non-visible faces by comparing the surface normal with the Z-vector in view space, just as it is possible to select concave patches and seperate meshs into convex hulls by comparing surface normals. Surface normals are produced by the cross product, which uses model space vectors. You can uniformally construct a complete surface basis by only a few simple vector operations using a surface primitive's spatial relations. That is why most back face culling algorithms don't even need to know surface normals, they can just have a look at vertex winding in view space. This conceptual perspective reveals massive opportunities. Why be limited to the evaluation of single primitives (i.e. backface culling => whole mesh => entire scene context)? Is there a virtual relation that can used be to model geometric holonymes? Must this system only evaluate geometric relations individually, or is there a relational symmetry which promises reduction to linear time?

Following these questions, you might develop a highly effective method for light transport. Anybody following what I said here?

Edit:

Oh.

https://en.wikipedia.org/wiki/Affine_connection

The terminology is due to Cartan and has its origins in the identification of tangent spaces in Euclidean space Rn by translation: the idea is that a choice of affine connection makes a manifold look infinitesimally like Euclidean space not just smoothly, but as an affine space.