Intro I have been working a while on this technology and since real-time raytracing is getting faster like with the Brigade Raytracer e.g., I believe this can be an important contribution to this area, as it might bring raytracing one step closer to being usable for video games.

I had a feeling, by watching their demos that they're doing this already. (but the video quality is very bad, it's hard to see, but the silhouette ghosting artifacts made me think that)

Algorithm: The technology exploits temporal coherence between two consecutive rendered images to speed up ray-casting. The idea is to store the x- y- and z-coordinate for each pixel in the scene in a coordinate-buffer and re-project it into the following screen using the differential view matrix. The resulting image will look as below.

you don't need to store x,y,z, it's enough to have the depth, from there and the screen pixel coordinates, you can re-project it. that's done e.g. in Crysis 2 (called temporal AA) and in killzone 4 it got recently famous ( http://www.killzone.com/en_GB/blog/news/2014-03-06_regarding-killzone-shadow-fall-and-1080p.html ) and old school cpu-tracer demos have done that as well. This becomes easily a memory bound thingy, that's why reducing the fragment size to a minimum is key. Practically it should be nearly not noticeable compared to the tracing time.

The method then gathers empty 2x2 pixel blocks on the screen and stores them into an indexbuffer for raycasting the holes. Raycasting single pixels too inefficient. Small holes remaining after the hole-filling pass are closed by a simple image filter.

in path tracing, the trick is to not only re-project the final pixel (those are anti-aliased etc. and give you anyway wrong result), you have to save the originally traced image (with 10spp it's 10x the size!) and re-project those samples, then you'll get a pretty perfect coverage.
updates are now done on an interleaved pattern, e.g. replacing 1 out of 10 samples of the re-projection source buffer per frame.

Results: Most of the pixels can be re-used this way as only a fraction of the original needs to be raycated, The speed up is significant and up to 5x the original speed, depending on the scene. The implementation is applied to voxel octree raycasting using open cl, but it can eventhough be used for conventional triangle based raycasting.

5x is also what I've seen for primary rays, if you have enough depth during path tracing, it can get closer to linear speedup (depending on the spp and the update rate).

Limitations: The method also comes with limitations of course. So the speed up depends on the motion in the scene obviously, and the method is only suitable for primary rays and pixel properties that remain constant over multiple frames, such as static ambient lighting. Further, during fast motions, the silhouettes of geometry close to the camera tends to loose precision and geometry in the background will not move as smooth as if the scene is fully raytraced each time. There, future work might include creating suitable image filters to avoid these effects.

this also works for secondary rays, but gets a bit more complex, you have to save not only the position, but also the 2nd bounce and recalculate the shading using the brdf. I think vray is doing that, calling it 'light cache'.

the trick with motion is that you won't notice artifacts that much, so you can keep the update-rate stable, areas of fast motions will look obviously worse, but you won't notice it that much. only real problem you can get is similar to stochastic rendering, where a lot of samples fall into the same pixel and you have to use some smart reconstruction filter to figure out which samples really contribute the that pixel, and which ones are just 'shining' through and should be rejected. that might not be noticeable that much in your video, but if you trace e.g. trees with detailed leaves, you'll have to re-project sky samples and close leave samples and there is not really an easy way to decide what sky pixel to reject and which ones to to keep. best I've seen so far was a guy who used primitive-ids and has done some minimal triangulation of samples with the same id, but the result were far from good.
in Crysis 2, during motion, you'll notice that tree-tops look like they get thicker, I guess they use simply a nearest filter to combine sub-samples.


the silhouette issue arises when you work on the final buffer, that's what you can see in the rasterizer versions like in Crysis 2, Killzone 4. re-projecting the spp-buffer (using the propper subpixel position) will end up with no ghosting on sillouettes (beside the previously mentioned reconstruction issue).

>>I had a feeling, by watching their demos that they're doing this already. (but the video quality is very bad, it's hard to see, but the silhouette ghosting artifacts made me think that)

Yes, a month ago they stated that. However, they dont tell if its used for primary or secondary rays and how the speedup was. I believe its simpler with secondary rays as you do a reverse projection, so you can avoid empty holes.

>>you don't need to store x,y,z, it's enough to have the depth

I tried to use the actual depth from the depthbuffer but that failed due to accumulated errors from frame to frame as most pixels are used over up to 30 frames.

>> Crysis 2 (called temporal AA) and in killzone 4 it got recently famous

TXAA (Crysis2) is just reusing the shading as I understand, so not for reconstructing the geometry,

The killzone method sounds more interesting as they predict pixels using the motion, so can reduce the render resolution.

I wonder if they need to store the motion vectors. Perhaps using the previous view matrices + depth is sufficient.

Sounds related to the approach used in mpg compression.

>> in path tracing, the trick is to not only re-project the final pixel (those are anti-aliased etc. and give you anyway wrong result), you have to save the originally traced image (with 10spp it's 10x the size!) and re-project those samples, then you'll get a pretty perfect coverage.

updates are now done on an interleaved pattern, e.g. replacing 1 out of 10 samples of the re-projection source buffer per frame.

10 samples sounds pretty memory consuming, but interesting to hear more details about that method.

>>the silhouette issue arises when you work on the final buffer, that's what you can see in the rasterizer versions like in Crysis 2, Killzone 4. re-projecting the spp-buffer (using the propper subpixel position) will end up with no ghosting on sillouettes (beside the previously mentioned reconstruction issue).

Apparently gamers also noticed the quality impact of this method , even its just applied to every second pixel. In my case the pixel is reused far longer which makes it even more difficult to keep the image consistent. I have also tried to increase the tile size, so every pixel on the screen is raycasted every 4th frame - that reduced the silhouette issue significantly, but also led to a lower performance obviously. Would be nice to track the silhouette over several frames somehow so it wont loose accuracy.