Intro I have been working a while on this technology and since real-time raytracing is getting faster like with the Brigade Raytracer e.g., I believe this can be an important contribution to this area, as it might bring raytracing one step closer to being usable for video games.
I had a feeling, by watching their demos that they're doing this already. (but the video quality is very bad, it's hard to see, but the silhouette ghosting artifacts made me think that)
Algorithm: The technology exploits temporal coherence between two consecutive rendered images to speed up ray-casting. The idea is to store the x- y- and z-coordinate for each pixel in the scene in a coordinate-buffer and re-project it into the following screen using the differential view matrix. The resulting image will look as below.
you don't need to store x,y,z, it's enough to have the depth, from there and the screen pixel coordinates, you can re-project it. that's done e.g. in Crysis 2 (called temporal AA) and in killzone 4 it got recently famous ( http://www.killzone.com/en_GB/blog/news/2014-03-06_regarding-killzone-shadow-fall-and-1080p.html
) and old school cpu-tracer demos have done that as well. This becomes easily a memory bound thingy, that's why reducing the fragment size to a minimum is key. Practically it should be nearly not noticeable compared to the tracing time.
The method then gathers empty 2x2 pixel blocks on the screen and stores them into an indexbuffer for raycasting the holes. Raycasting single pixels too inefficient. Small holes remaining after the hole-filling pass are closed by a simple image filter.
in path tracing, the trick is to not only re-project the final pixel (those are anti-aliased etc. and give you anyway wrong result), you have to save the originally traced image (with 10spp it's 10x the size!) and re-project those samples, then you'll get a pretty perfect coverage.
updates are now done on an interleaved pattern, e.g. replacing 1 out of 10 samples of the re-projection source buffer per frame.
Results: Most of the pixels can be re-used this way as only a fraction of the original needs to be raycated, The speed up is significant and up to 5x the original speed, depending on the scene. The implementation is applied to voxel octree raycasting using open cl, but it can eventhough be used for conventional triangle based raycasting.
5x is also what I've seen for primary rays, if you have enough depth during path tracing, it can get closer to linear speedup (depending on the spp and the update rate).
Limitations: The method also comes with limitations of course. So the speed up depends on the motion in the scene obviously, and the method is only suitable for primary rays and pixel properties that remain constant over multiple frames, such as static ambient lighting. Further, during fast motions, the silhouettes of geometry close to the camera tends to loose precision and geometry in the background will not move as smooth as if the scene is fully raytraced each time. There, future work might include creating suitable image filters to avoid these effects.
this also works for secondary rays, but gets a bit more complex, you have to save not only the position, but also the 2nd bounce and recalculate the shading using the brdf. I think vray is doing that, calling it 'light cache'.
the trick with motion is that you won't notice artifacts that much, so you can keep the update-rate stable, areas of fast motions will look obviously worse, but you won't notice it that much. only real problem you can get is similar to stochastic rendering, where a lot of samples fall into the same pixel and you have to use some smart reconstruction filter to figure out which samples really contribute the that pixel, and which ones are just 'shining' through and should be rejected. that might not be noticeable that much in your video, but if you trace e.g. trees with detailed leaves, you'll have to re-project sky samples and close leave samples and there is not really an easy way to decide what sky pixel to reject and which ones to to keep. best I've seen so far was a guy who used primitive-ids and has done some minimal triangulation of samples with the same id, but the result were far from good.
in Crysis 2, during motion, you'll notice that tree-tops look like they get thicker, I guess they use simply a nearest filter to combine sub-samples.
the silhouette issue arises when you work on the final buffer, that's what you can see in the rasterizer versions like in Crysis 2, Killzone 4. re-projecting the spp-buffer (using the propper subpixel position) will end up with no ghosting on sillouettes (beside the previously mentioned reconstruction issue).