I think you misunderstood what i meant.
With your further explanation it seems clearer that you are taking a wrong turn.
but that adds rather complicated problems (with materials,textures,shaders etc).
Materials (which includes textures and shaders) are no more a problem when using octrees than if you simply had a linear array of objects and rendered them naïvely. They have absolutely no relationship with the culling process.
Even then if a Terrain AABB intersects the frustum i will have to draw the entire thing which can be humongus, still drawing invisible parts of it.
Terrain is always divided into chunks (usually of, say, 257×257 vertices, smaller with LOD) and visible chunks are drawn.
Also I realize that the entire scene can be stuffed inside one octree and culled from there
This is the correct route to take. Your complications related to materials are fictional.
They are Partly visible (their bounding volume is bigger than the frusum but still visible - Terrains,Big Buildings etc...)
All large objects are subdivided. Terrain into chunks, buildings into rooms and halls, interiors and exteriors, each with LOD’s, etc. Further special-case culling takes place inside buildings to avoid drawing the exterior altogether (if you can’t see outside of the building then the sky, terrain, and all outdoor objects are immediately skipped and a second, smaller octree is used which spans only the inside of the building).
The idea is that i have some kind of a BVH(maybe octree of AABBs)
A BVH and an octree are separate structures—an octree is not a kind of BVH.
There is nothing wrong with how you plan to structure objects using multiple smaller objects. A model is made up of multiple meshes. Each can have its own AABB, plus the AABB for the entire model. And if the model’s AABB is entirely inside the frustum there is no need to test the smaller mesh AABB’s. This is standard.
But culling itself is not a free series of instructions. If your culling method is too complex then it becomes slower than not culling at all. It is better to check mesh AABB’s (the smaller parts of models) linearly one-by-one than to maintain, update, and traverse yet another octree.
There exists the scene octree (or BVH) and nothing more (unless you add the interior/exterior special case for being inside buildings mentioned above). There are not mini-octrees inside all of the models, as such a costly culling method for so few objects doesn’t make sense. They are
however culled. Just one-by-one, no secondary acceleration structures (a BVH here might
make sense only for static objects, but again you would need to test that it actually helps, because you are entering a granularity where it becomes questionable).
And finally, none of this has anything to do with materials. I don’t know why you even mentioned them. Culling and rendering are disjointed and separate operations. You build a list of renderables via the culling pass, then you sort them with a render queue, then you render them. Culling uses an object’s AABB, and an AABB has no idea what materials are, it simply defines a geometric space.
I think thats what i am looking for. Using 1 VBO (which i already have prebuilt) and only extract the vertices i currently need at each test against the tree. Can you explain further , maybe with some codeblocks how would that work ? Thanks !
This is unrelated to culling, but a model has a single VBO and each mesh has an IBO which references vertices inside of it. This isn’t a culling issue, but simply a good idea to avoid rebinding VBO’s etc. Based on your explanation, it doesn’t seem worth it to rebuild the IBO(’s) in any way related to culling. Subdividing meshes at run-time is overkill and simply not worth the cost just to avoid drawing a few off-screen triangles.
Edit: What structure should i use to store the bounding boxes of the objects . I assume i can group the boxes in huge chunks and test against that and discard the entire chunk if not visible . But how would one build that structure ?
Models and meshes have AABB’s. The AABB’s inside the models (the mesh AABB’s) are too numerous and small and should be ignored for this part of the discussion, as including them in the scene’s octree or BVH would cause update hell (unless they are purely static).
The scene’s spacial-partitioning scheme can be either a BVH or an octree, but care must be taken in how either is implemented for there to be any benefit. In either case, the finest granularity is at the model level, not the mesh level.
In the case of octrees, the nodes in the tree must be linear in memory. Cache misses while traversing the tree will otherwise become so significant that the octree itself causes performance to drop overall, even if half the objects in the scene are ultimately culled.
Additionally, octrees are memory-consuming and in a naïve implementation you can only afford to have a few levels in the tree. This is why a hybrid approach is more common, using an Efficient Instant-Insertion Quadtrees
for the X and Z (horizontal plane) and a stack for the Y (vertical plane).
In the case of BVH’s performance will depend heavily on your construction strategy, and they can’t be rebuilt at run-time efficiently (can only contain static objects).https://en.wikipedia.org/wiki/Bounding_volume_hierarchy#BVH_design_issues
Note that it is perfectly acceptable to have terrain be a special case and be culled differently from the rest of the scene. In fact terrain culling can be so wildly different that it can define the terrain itself.http://research.microsoft.com/en-us/um/people/hoppe/gpugcm.pdfhttp://www.flipcode.com/archives/article_geomipmaps.pdf