I've been working on making my own OpenGL rendering engine for first-person based game.

For simplicity's sake, let's assume I'm rendering a large static scene which contains both opaque and transparent objects.

From what I understand, I have to sort my transparent objects back to front in order for them to be rendered correctly.

If I heard correctly, BSP trees are the way to do this as long as my scene is static.

However, BSp trees seem to sort the individual polygons in my scene, rather than the individual objects.

Is this correct? If not, how do I get the correct "object order" for objects in my scene.

I have only constructed a basic rendering loop in which I store all my geometry in one large VBO, which I then draw in one go using indexed drawing.

How would I be able to tell OpenGl in which order it should render the individual polygons, without having to buffer my vertex/index data over and over again?

If anyone could shed some light on this, or point me in the direction of some good examples using modern OpenGL, I would be forever in your debt ;-).

However, BSp trees seem to sort the individual polygons in my scene, rather than the individual objects.

Is this correct?

BSP doesn't care about what you store in its nodes. It might be single polygon, or groups of objects, or whatever you might want to sort.

how do I get the correct "object order" for objects in my scene.

You check for sign of dotproduct between splitting plane's normal and cam's direction vector.

Depending on that sign you decide which next node to traverse first - either front or back half-space.

I just iterate through everything (game objects) I know that I know are visible and throw them into a separate binary tree (every frame) using their squared distance as each node's value with a pointer to the object. Then when I render, I just walk through the binary tree from furthest to nearest. FYI: The reason I used a squared distance to populate the tree is to avoid using a sqrt.

A BSP is not the preferred method for rendering translucent objects.

Use a standard render-queue and sort full objects (sub-meshes) back-to-front.

Ok, so basically: Just take your translucent (sub)meshes, order them back to front and render then in that order. But exactly how do I correctly order them back to front in a reliable manner if BSP trees can't be used?

I've read on or two books on OpenGL and game engine development, but some basic concepts about scene management, rendering phases or other practical approaches for rendering a complete scene always seem to be missing.Can anyone point me to any good resources that tackle these problems specifically?

This article talks about using dynamic objects with BSP trees: http://web.cs.wpi.edu/~matt/courses/cs563/talks/bsp/document.html

Ok, so basically: Just take your translucent (sub)meshes, order them back to front and render then in that order. But exactly how do I correctly order them back to front in a reliable manner if BSP trees can't be used?

Depends..

Firstly, for it to work entirely correct your submeshes must be convex and non-intersecting (or they would need to have their polygons sorted too for perfect close-ups), though if they're nearly so you might get away with it anyway. Then after that you could sort based on Z or distance to the camera, from the closest point on the objects bounding-sphere/box.

Thanks, sounds easy enough. I'll try to avoid using concave translucent objects for now and just focus on getting everything working with a simple back to front ordering mechanism based on the distance to the camera.

If you draw planar stuff like windows it can be good to use something like a BSP, for example in the attached image with an orange and a blue window. If you draw more compact objects it's not an issue.

You probably don't have to worry about this case, but for static geometry then everything on the close side of the object's plane should be drawn before everything on the far side. So in this case either the blue window has to be drawn first because it's on the same side of the orange window as the camera, or the orange window would have to be split in two objects, along the blue window plane, and the half on the same side of the blue plane as the camera would be drawn first, then the blue window, then the part on the back side.

That's the basic theory of the BSP for static geometry, the geometry is preprocessed to split all objects that cross another object plane, so in the actual game it can never happen because the objects never cross each other. That's also why they're not used much anymore as there are often better ways for such high polygon models as are used nowadays. In Quake 1 it made perfect sense as there were only a few planes.

A BSP is not the preferred method for rendering translucent objects.

Why not? Isn't this the main (or first) reason BSP was invented - to allow for fast back-to-front traversal of arbitrary geometry based on camera position, without having to sort the geometry every frame?

The only problem with BSPs is that they're expensive to build in real-time. But for static geometry, for which the BSP can be built offline, aren't they still the best choice?

And unless other techniques like PVS or occlusion culling (which actually proved to be inefficient on hardware unless actually coupled with space partitioning) are also used, I'm still not convinced that sorting even dynamic geometry (convex or not) in real-time is any faster than generating BSPs in real-time.