Hi all!

We've implemented a hybrid shadow algorithm where shadow mapping gives us some boundary pixels which need to be further checked. One of the ways to check whether this boundary pixel is in shadow (or not) is to shoot a ray towards the light and check for intersection with the scene's triangles.

Currently, we're thinking of storing the scene in an octree (in the texture memory of the GPU). Then shoot rays from every pixel and check for intersection by traversing the octree in the fragment shader.

Can you point me to some tutorials/papers regarding such intersection detection? I couldn't find much information about how the octree should be stored in the texture, and how the octree should be traversed inside the fragment shader etc..