Engine design questions - GameDev.Net Discussion Forums

Add Forum to Favorites | Send Topic To a Friend | View Forum FAQ | Track this topic

Last Thread Next Thread

Engine design questions

rogierpennink Member since: 4/29/2006 From: Castricum, Netherlands

Posted - 11/17/2008 8:08:24 AM

Thus far I've found the gamedev forums, and this forum in particular, to provide very useful information on everything that has to do with game engines. I've read many 'classic' threads where Yann L. explains everything from shadow mapping to scene graph implementations, and they've proven very helpful.

However, in the process of working on my own rendering, I'm finding it difficult to ty all these theories and 'best designs' together in a coherent design that makes sense for me. I'll list the issues I'm running into and my thoughts about them one by one below...

Also note that I'm using OpenGL, and though the renderer is designed to easily allow support for other libraries such as Direct3D I see no need to this yet, so I may sometimes refer to OpenGL specific things.

The scenegraph
Everything I've read about the subject has led me to believe that spatial ordering and other optimization techniques don't belong in a scene graph. Currently I have a very limited SceneGraph class that consists of GroupNode and GeometryNode objects only (both are derived from SceneNode). GroupNodes can have children, GeometryNodes can't have children, so all geometry nodes are leaves of the tree.

This works exactly the way I thought a scene graph is supposed to work (I can add parent-child relations between objects in a scene, and transformations in each node are relative to the parent node). Though many threads I've read about scene graphs add more types of nodes, I probably won't be doing that in the near future because I want things to remain as simple as possible until I've grown more comfortable with the design.

So far so good, but my understanding starts lacking when it comes to passing objects from the scene graph to the Renderer. This has to do with the way in which GeometryNodes are defined in my engine:
- GeometryNode
  The only thing that makes a GeometryNode different from its parent, SceneNode, is that it contains a BaseGeometry object. The BaseGeometry class is the parent class for all geometry in the engine. So you can make a BoxGeometry object, for example, that inherits from BaseGeometry and which defines the geometry for a box. I'm also working on classes like 'CaligariGeometry', or '3DSMaxGeometry' that will load geometry from disk. Similarly, I imagine there would be TerrainGeometry objects. Because all these Geometry classes inherit from BaseGeometry, they can all be passed to GeometryNodes in the scene graph. The scene graph doesn't really care what the geometry represents, it only knows about GeometryNodes, but what they contain is not of the scene graph's concern.
- BaseGeometry
  BaseGeometry, as said, contains geometry data. I currently distinguish between many types of buffers. Think IndexBuffer, VertexBuffer, NormalBuffer, TexcoordBuffer etc. The reason why I did this is because I'm using the OpenGL buffer objects extension, and the framerate was higher when I used this system of many buffers for one object, rather than one large buffer with Vertex objects that contain everything from vertex position data to texture coordinates. I'm quite sure my tests are to be taken too seriously as I've never used any substantial number of vertices. Thus far all I'm rendering is a 100x100 grid with a couple of boxes on it, because I cannot load arbitrary 3D file formats yet that would allow me to more easily load more data.
This is actually my first problem. With so many buffer objects for each (potentially small) geometric entity I foresee the rendering being horribly inefficient. Suppose you render 10,000 boxes with this scheme. Every box has 8 vertices, 8 colours, and 36 indices. That means this scene requires the use of 30,000 buffer objects whose contents all range from 8 to 36 indices. I thought it would be more efficient if the Renderer class kept several large buffers itself, but as far as I know that means you'd have to copy the geometry data into the renderer's buffers every frame (because with culling etc. going on I can't be certain if a geometric entity is still supposed to be in the renderer's buffers or not), which defeats the point of using vertex buffer objects altogether.

The second thing I have a hard time understanding how culling comes into play. It seems that using a scene graph for culling is not good; a scene graph is not meant for that. Reading through one of the threads on gamedev.net where Yann L. explains all about terrain and scene graphs, I got the impression that it may be good practise to let geometry cull itself. I can see the advantages of this in for example, a TerrainGeometry class, where the TerrainGeometry would implement a spatial partitioning scheme. This way the scene graph will still think of the terrain as a single "GeometryNode" entity, but the terrain itself could be a quadtree without the scene graph ever knowing.

For terrain it'd work well, but other entities? Would it work to have an OctreeGeometry as a base class and have, for example, classes like 3DSMaxGeometry also inherit from OctreeGeometry (next to BaseGeometry)? I'm guessing this won't work as well... Animated models aren't really 'optimised' by octrees (from what I've read), and I can't see the benefit of putting small static geometry in an octree either. An other solution could be to work with several 'phases', first of which would be to update the scene graph, and then send all geometry to a culling phase. The drawback I can see to this is that all geometry will be treated equally, which I assume to not be desirable in all cases. Culling algorithms are going to be different for different types of geometry, so some distinction should still be made.

These are just some of the questions I'm facing, and perhaps I shouldn't even bother with them and proceed making a horrible inefficient renderer that is to be improved upon later, but I'd still like to hear some opinions, as I find it nearly impossible to 'move on' without addressing any of these concerns. I constantly fear that I'm making my engine so inefficient that the second I'm done with it, I'll want to scratch half of it. Eventually, facing these questions will be inevitable so I thought I might as well do it now.

Thanks in advance for any feedback :)

User Rating: 1198 | Rate This User

Report this Post to a Moderator | Link

Ingenu Member since: 1/3/2000 From: Maisons-laffitte, France

Posted - 11/17/2008 10:14:20 AM

I'm gonna do some shameless advertising first ;)

I've written a series of articles about engine architecture, intended for beginners here :
http://www.beyond3d.com/content/articles/98 (Part 1)
http://www.beyond3d.com/content/articles/102/ (Part 2)
(More written, but not published yet :( )

You might find something useful in there.

I tend to rename things, giving good names is difficult, but I find important to name things as accurately as possible in order to be understood easily.
So in my engine design (not necessarily the best, it's just one solution to the problem, probably not the worst at least ;) ), I have a SceneTree (a Tree is a Graph, but everyone gets what a tree is w/o any further specifications), a SpatialGraph (Directed Acyclic Graph), and I used to have a RenderGraph, but I replaced it by a kind of RenderQueue.

The SceneTree is solely used for hierarchical animation. (Be it skeletal animation or a sword held in a character's hand.) This node type has an update(...) function that allows animation, and the nodes make a tree (a single parent, any number of children).

The SpatialGraph is used for culling (in fact finding what's visible). It's its only purpose in life to make culling fast.

The RenderQueue, is filled during culling in the spatialgraph. It's not a single array though, it's a little more complex than that. I did write something directly inspired from Yann Lombard thread about his material system.
The only task of this RenderQueue is to render whatever has been found visible FAST. (ie means sorting sub arrays per key, each key holding data such as depth, shader ID...; whatever is appropriate.)

Examples:
Terrain
It has a single node in the SceneTree, that does nothing.
(Nothing happens based on time on this terrain.)
It's made of a number of nodes in that SpatialGraph, making a Quadtree, which root is the Terrain's SpatialNode, and Leaves would contain mesh or pointers to meshes.

Character
It has a number of nodes in the SceneTree, for Meshs and Bones, which both might be animated hierarchicaly. It might also have something attached to his hand(s), that would need hierarchical relationship.
It has one SpatialLeaf node per mesh.

I call Geometry a Mesh instead.
I might be a bit synthetic, ask if I'm not clear enough.

[Edited by - Ingenu on November 20, 2008 5:14:29 AM]

-* So many things to do, so little time to spend. *-

User Rating: 1061 | Rate This User

Report this Post to a Moderator | Link

wolf XNA/DirectX MVP Member since: 1/8/2000 From: Carlsbad, CA, United States

Posted - 11/17/2008 1:15:06 PM

I very much like the SceneTree, SpatialGraph and RenderView nomenclature. It says what it does and this is a huge advantage over the word SceneGraph. I will use Ingenu terms from now on.

http://diaryofagraphicsprogrammer.blogspot.com/
Check out our online D3D10 book: Programming Vertex, Geometry, and Pixel Shaders

User Rating: 1469 | Rate This User

Report this Post to a Moderator | Link

smr GDNet+ Member since: 7/16/2004 From: Pekin, IL, United States

Posted - 11/17/2008 3:55:31 PM

If all of the boxes are the same, then why would you need so many vertex buffers? Can't you reuse them?

User Rating: 1506 | Rate This User

Report this Post to a Moderator | Link

Paool Member since: 11/30/2007 From: Wroclaw, Poland

Posted - 11/20/2008 3:26:20 AM

It's very hard to think of some kind of general 3D engine architecture which can handle everything in a nice object-oriented way and is also extremely fast and efficient. If something is for everything - it's for nothing.

Why not to keep all subsystems separately? Terrain could store all its nodes in quad-tree structure, characters could have its own hierarchy tree, and world geometry might be handled/occluded by its own BSP/PVS, or oct-tree structure (depending on what we need).
All subsystems write renderable items into render manager (render queue), which perform sorting by depth, material ID, priority, or whatever we want.

Using this approach we can optimize every single subsystem without touching the others.

User Rating: 1027 | Rate This User

Report this Post to a Moderator | Link

Jason Z GDNet+ Member since: 2/4/2004 From: Royal Oak, MI, United States

Posted - 11/20/2008 6:26:57 AM

I suppose I'll throw my design into the discussion too, although it doesn't deal with the scene graph side of things as much as you are looking for:

Hieroglyph Design

You don't actually need to have many buffers to represent many objects. You can simply have a single buffer (or multiple large ones if you like) and each object indexes into that big buffer instead of each object having its own. Then you just need to centralize the access to the buffer, which can be coordinated through your renderer interface.

Jason Zink :: MVP XNA/DirectX

"Intellectuals solve problems, geniuses prevent them." - Albert Einstein
Check out my game: Lunar Rift :: Dual-Paraboloid Mapping Article :: Parallax Occlusion Mapping Article :: Fast Silhouettes Article
Check out our free online D3D10 book: Programming Vertex, Geometry, and Pixel Shaders

User Rating: 1488 | Rate This User

Report this Post to a Moderator | Link

All times are ET (US)

Last Thread Next Thread

Forum Rules:

You may not post new threads
You may post replies
You may not edit your posts
You may not use HTML in your posts

Jump To: