What information do you need from the intersection, just the point or the triangle index?

I am currently trying to figure out what would be the best/fastest method to get information

from a triangle/ray intersection and hit point calculation done in GLSL.

To go back to this, there really is no fast way to do this. One way you could do this is to put a camera at the start point of the ray and then look down the ray. Then render the cube, glReadPixels(centerScreenPixel). Then you have the depth from where the center of the camera hit. You can use glUnproject() to get the actual position it hit.

That is of course terrible, I think all ways relative to a graphics card performing raycasting is bad. Compute Shaders would be the way to go for GPU. Otherwise just get download a physics engine or a raytracing engine like Intel Embree, and do a raycast on the CPU. All bullets in big game worlds are done on the CPU and they have many more than 10,000 triangles.

I would look at getting a physics engine integrated(PhysX, Bullet, Newton) and using a cpu side raycast.

I've tried using the Transform Feedback, creating a buffer to store if a hit is present and another to store a world space hit locations.
It works great, but the glGetBufferSubData(), to get data back to the app, is just so expensive, nearly 0.8ms just for that call, all other work is lighting fast.

Not sure if you tried it, but keep in mind that TF (AKA stream out) can write a varying arount of data to a buffer. So your GS can either not output anything, or output hit depth and triangle index, which would greatly reduce the amount of data to be read back.

Depending on thr number of triangles in the mesh and the accuracy required, you could also use a second pass on the GPU to find the minimum-depth element in that TF buffer. Use a VS that outputs every TF-buffer entry at a hard-coded point location in the middle of a 1x1 pixel render target of 32bit unit format. Use a PS that outputs fixed point depth in the high bits and triangle ID in the low bits (e.g. 16&16). Set the blend mode to MIN (and pre-clear the texture to 0xFFFFFFFF) to get the smallest depth result, along with the corresponding triangle ID encoded in its low bits. Then there's only a single pixel texture to read back to the CPU.

Now though, read backs, no matter how small, are always stupidly expensive if done on the same frame as the commands that produced the data. To maintain high throughput, GPUs have very high command latency and very long pipelines. This latency is usually one to three frames, or in the dozens of milliseconds. Whenever you issue a GL command and immediately try to read back the results, you're asking the driver to flush this long pipeline and stall until it's empty. Not only will this take a dozen ms, but you're got a lovely pipeline bubble when you start using the GPU again afterwards.

To avoid this, you have to also pipeline your GPU queries, such as with double buffering. Ever frame you issue GL commands to generate a result into resource A, and then attempt to read back the results from last frame that are in resource B. Hopefully this avoids flushing the pipeline. To get more certainty, add more frames of buffering to make your pipeline longer...

I don't remember the GL details off the top of my head, but in D3D you can make DEFAULT resources that live on the GPU, and staging resources that live on the CPU. You would issue commands to generate data into GPU_A and commands to copy GPU_A into CPU_A. A frame later, you'd then map CPU_A to read your result.


As above though, why not just use a CPU raytrace? Instead of brute forcing the answer, you'd use a BVH/etc to quickly discard most triangles and then only check a handful.

I am rendering a dynamic terrain using displacement mapping and viewpoint LOD on tessellation.

Well you want to get away from using displacement that pushes vertices really high away from their triangle, this way you can just raycast against the triangle, then find what pixel maps to that location of the triangle and then offset it. To get the exact displacement depends on your setup. You can always raycast against the actual heightmap of the terrain (that is what I do). You always just raycast againt the heightmap, not the tessellation detail. Again, physics engines usually have a raycast heightmap function which will do this for you. Tessellation is still based upon the height map.

Now if you were tessellation really strangely and the triangles were shifting so much that a hillside moves down drastically, when then yes, you could have something floating above that part of the hill, but your other option is that you have a ton of objects popping up and down all the time.