Abstracting graphics libraries from the game engine
Hi all
In a game engine where DirectX/OpenGL are abstracted away into dlls and used via interfaces, what generally happens with things like native vector classes and the various helper methods (e.g. everything in d3dx9math.h).
My initial thought was that anything that uses, say, D3DXVECTOR3 or any helper classes would have to be contained/encapsulated within a MyEngineDX9.dll/MyEngineOpenGL.dll library and everything outside of that in the core parts of the engine wouldn't need to know anything about vertices or planes, etc as they'll be dealing with rendering at a higher level. At the moment, I find it hard to believe I won't need a vector class of some description outside of the graphics library, but I can't really see a way of abstracting away the D3DXVECTOR3 struct, or any of the other helper methods.
For example, my terrain rendering engine uses D3DXVECTOR3 and other DX9 helper methods a fair amount - so does that mean it should go into the MyEngineDX9.dll/MyEngineOpenGL.dll libraries? (obviously the OpenGL version would use OpenGL's native types).
Has anyone dealt with this situation before that could possibly offer any advice?
Thanks
It is a tricky one that. I think normally people use their own vector type. You can also cast the D3DXVECTOR3 to a struct of 3 floats. Ultimately the answer probably lies in a stream somehow for complete abstraction but that is probably going too far.
A common approach is a low-level "core" or "math" library shared by higher level components of the game engine such as graphics and physics libraries. This requires conversion or at least casting from, for example, MyVector3 to D3DXVECTOR3 in your DX9 library. At least that's what I've seen working at medium to large game companies.
It's a surprising amount of tedious work to support multiple video hardware APIs such as Direct3D and OpenGL (and LibGCM and so on). If you're not getting paid to do it, why not spend the effort on something more interesting and useful like making a cool little indie game that I can play... :)
It's a surprising amount of tedious work to support multiple video hardware APIs such as Direct3D and OpenGL (and LibGCM and so on). If you're not getting paid to do it, why not spend the effort on something more interesting and useful like making a cool little indie game that I can play... :)
I would definitely consider BFG's advice before making a final decision: make a game first and then later, maybe worry about something like this. After making a small game or two you'll have more perspective on what is necessary in a generic graphics library, which generally leads to better design.
If you decide to go this route, I'd suggest really thinking about this in-depth before you even start touching any code. As mentioned previously, there are usually some core libraries that back up such a system, such as a general math library that can easily lend itself to specifics. What you probably want to do is take out any library-specific utility calls and focus on the bare necessities.
I'm no expert in this area, and haven't done anything myself, but I've thought about it from time to time. Anyways, best of luck with whatever you decide!
If you decide to go this route, I'd suggest really thinking about this in-depth before you even start touching any code. As mentioned previously, there are usually some core libraries that back up such a system, such as a general math library that can easily lend itself to specifics. What you probably want to do is take out any library-specific utility calls and focus on the bare necessities.
I'm no expert in this area, and haven't done anything myself, but I've thought about it from time to time. Anyways, best of luck with whatever you decide!
Thanks for the responses guys.
I definitely agree that writing a game or two first would assist in the evolution of the engine but I didn't want to get down the line suddenly needing it to be platform/graphics independent and to have put no planning into that area at all. At least if I can plan the framework and create it in the mindset that at some stage things might need to be abstracted away, it'll be better prepared for it. I do want to avoid the situation where indecision in engine design halts the game development completely.
I do have a math library (or at least a skeleton of such) in my core layer so I think what I'll do is try and abstract away the native classes in there, along the lines of what BFG suggested, or if I'm feeling particularly mathsy, roll my own. Having said that though, I'm not really using a lot of the helper functions supplied in d3dx9math.h so I could perhaps just write them myself in the maths library as I need them. I already have my own classes and helpers for planes and frustums, etc. And the type most used by the graphics library is only likely to be vectors anyway.
I don't doubt what you're saying for a second!
I definitely agree that writing a game or two first would assist in the evolution of the engine but I didn't want to get down the line suddenly needing it to be platform/graphics independent and to have put no planning into that area at all. At least if I can plan the framework and create it in the mindset that at some stage things might need to be abstracted away, it'll be better prepared for it. I do want to avoid the situation where indecision in engine design halts the game development completely.
I do have a math library (or at least a skeleton of such) in my core layer so I think what I'll do is try and abstract away the native classes in there, along the lines of what BFG suggested, or if I'm feeling particularly mathsy, roll my own. Having said that though, I'm not really using a lot of the helper functions supplied in d3dx9math.h so I could perhaps just write them myself in the maths library as I need them. I already have my own classes and helpers for planes and frustums, etc. And the type most used by the graphics library is only likely to be vectors anyway.
Quote:It's a surprising amount of tedious work to support multiple video hardware APIs such as Direct3D and OpenGL (and LibGCM and so on)
I don't doubt what you're saying for a second!
Quote:Original post by RobMaddisonPersonally, I don't think it's worth your time to think about how something may need to be abstracted in the future. No matter how well you plan, you're going to find that you didn't quite plan as well as you'd have liked.
I definitely agree that writing a game or two first would assist in the evolution of the engine but I didn't want to get down the line suddenly needing it to be platform/graphics independent and to have put no planning into that area at all. At least if I can plan the framework and create it in the mindset that at some stage things might need to be abstracted away, it'll be better prepared for it. I do want to avoid the situation where indecision in engine design halts the game development completely.
My suggestion is to follow the "YAGNI" principle which states, basically, that if you don't need something right now, don't bother trying to implment functionality that may "support" it in the future.
In addition to that, I'm not generally a fan of the "DirectXEngine.dll and OpenGLEngine.dll" approach. The thing is, you'll never have a need to switch rendering APIs at runtime and so encapsulating them in a DLL is rather pointless - particularly using runtime polymorphism to encapsulate them (that is, a base "Renderer" class with "DirectXRenderer" and "OpenGLRenderer" sub-classes) (This is probably my biggest beef with Ogre). Much better, in my opinion is to make use of compile-time polymorphism - choose your renderer at compile time. Typically, the only reason you'd want to support both DirectX and OpenGL is if you want portability on one hand (and hence need OpenGL on Mac/Linux) and DirectX on Windows on the other hand (where driver support, etc, is much better). In that case, you're going to be compiling different version for Mac/Linux/Windows anyway.
With that in mind, if you have a "core" library that contains all your math routines, you can simply code two versions: one with a DirectX back-end and one with an OpenGL backend, as long as the interface is the same, you can just switch between them at compile time.
Edit: actually, since OpenGL doesn't really have any math functions built in, you're going to have to code all of that stuff manually anyway if you choose to support OpenGL. In that case, I suggest you simply use the D3DX functions now (since they work, they're fast, and so on), and if you decide you want to support OpenGL in the future, you can refactor your math routines to not use the D3DX math stuff anymore at that point...
Ah, no I wasn't going for runtime switching, I agree about how pointless that is. My core startup library determines which platform to use (win32, Linux, etc), loads that library which in turn loads the relevant graphics library at runtime based on a command line switch, config entry or whatever. I then just use common interfaces to platform and graphics-dependent libraries so the game engine core doesn't care which library it's using, but it knows there's only one. This approach can, of course, easily adapt to compile time linking, so MyGameWin32DX9.exe, or similar, is possible.
I agree with you on the YAGNI approach but in my case, I'm fairly sure of the requirement to cater for a different graphics/platform library, so I at least need to so some preliminary planning. lf, for no other reason, I'd like my engine design to be clean and componentised.
I really appreciate your thoughts and comments, it's great to get another perspective on what I'm planning.
Thanks.
I agree with you on the YAGNI approach but in my case, I'm fairly sure of the requirement to cater for a different graphics/platform library, so I at least need to so some preliminary planning. lf, for no other reason, I'd like my engine design to be clean and componentised.
I really appreciate your thoughts and comments, it's great to get another perspective on what I'm planning.
Thanks.
For sure you have to ban all platform specific code from the common code parts if you plan for multi platform support. Where to draw the broder between platform specific and common code is the real problem then, as you already have encountered it.
I personally do platform abstraction in multiple sub-systems. Those are for now Storage for file access, Video for gfx card/monitor combinations, Audio for sound hardware, Graphics for graphics rendering, Sound for sound rendering, Input for input handling, and Thread for multi-threading. Herein the Graphics sub-system is the one of interest for you, because it wraps OpenGL or D3D or whatever.
Stuff that does not closely relate to one of the sub-systems is implemented in a common way. A math library counts to that, too. This library supports vectors, matrices, quaternions, transformations, regions, and similar low level stuff. W.r.t. compatibility the layout of vectors and matrices is oriented on the layout needed by OpenGL and D3D (well, this is no problem for vectors at all, but it may have been one for matrices; fortunately, although OpenGL and D3D use a different vector notation, they also use a matching coefficient order, so that the overall layout of matrices in memory is the same for both).
Now, I handle definitions of meshes more like a set of data rather than a structure of vectors, and so do I for terrain. In fact, this resembles the vertex buffer idea. The layout of vertex buffers can be pre-defined, or it can be negotiated between the mesh and the Graphics. Nevertheless is the data layout again compatible to work with Vector and Matrix classes (keyword "data binding").
I personally do platform abstraction in multiple sub-systems. Those are for now Storage for file access, Video for gfx card/monitor combinations, Audio for sound hardware, Graphics for graphics rendering, Sound for sound rendering, Input for input handling, and Thread for multi-threading. Herein the Graphics sub-system is the one of interest for you, because it wraps OpenGL or D3D or whatever.
Stuff that does not closely relate to one of the sub-systems is implemented in a common way. A math library counts to that, too. This library supports vectors, matrices, quaternions, transformations, regions, and similar low level stuff. W.r.t. compatibility the layout of vectors and matrices is oriented on the layout needed by OpenGL and D3D (well, this is no problem for vectors at all, but it may have been one for matrices; fortunately, although OpenGL and D3D use a different vector notation, they also use a matching coefficient order, so that the overall layout of matrices in memory is the same for both).
Now, I handle definitions of meshes more like a set of data rather than a structure of vectors, and so do I for terrain. In fact, this resembles the vertex buffer idea. The layout of vertex buffers can be pre-defined, or it can be negotiated between the mesh and the Graphics. Nevertheless is the data layout again compatible to work with Vector and Matrix classes (keyword "data binding").
I have a math.lib, which has my custom vectors/matrices/planes/octrees/quaternions/polygon classes, etc... then if d3d is used I cast them into d3dxvector/d3dxmatrix, etc... (same for the ogl natives version).
pros:
- i write all custom maths code myself on my math classes(collision, intersection, rotations etc... sorry i love math); this way I can test that my math.lib is correct and view the result without d3d or ogl.
(I only pass the result to d3d or ogl for rendering)
- you can parameterise as much as you want
- there's a lot of info out there these days to help you implement good maths
- you learn more on maths applications and how it all works
cons:
- if u don't love math, it's tedious
- takes a while to get right (ONLY if you got onto the wrong help/tutorial/article or misunderstood some math explanation)
- long research
I'm sure there are more pros and cons, but if your priority is writing a game, then do that first (but a good future proof design requires that you think of such before later, and good thinking requires ample time & thoughts. I think :-))
pros:
- i write all custom maths code myself on my math classes(collision, intersection, rotations etc... sorry i love math); this way I can test that my math.lib is correct and view the result without d3d or ogl.
(I only pass the result to d3d or ogl for rendering)
- you can parameterise as much as you want
- there's a lot of info out there these days to help you implement good maths
- you learn more on maths applications and how it all works
cons:
- if u don't love math, it's tedious
- takes a while to get right (ONLY if you got onto the wrong help/tutorial/article or misunderstood some math explanation)
- long research
I'm sure there are more pros and cons, but if your priority is writing a game, then do that first (but a good future proof design requires that you think of such before later, and good thinking requires ample time & thoughts. I think :-))
Quote:Original post by Codeka
In addition to that, I'm not generally a fan of the "DirectXEngine.dll and OpenGLEngine.dll" approach. The thing is, you'll never have a need to switch rendering APIs at runtime and so encapsulating them in a DLL is rather pointless - particularly using runtime polymorphism to encapsulate them (that is, a base "Renderer" class with "DirectXRenderer" and "OpenGLRenderer" sub-classes) (This is probably my biggest beef with Ogre). Much better, in my opinion is to make use of compile-time polymorphism - choose your renderer at compile time. Typically, the only reason you'd want to support both DirectX and OpenGL is if you want portability on one hand (and hence need OpenGL on Mac/Linux) and DirectX on Windows on the other hand (where driver support, etc, is much better). In that case, you're going to be compiling different version for Mac/Linux/Windows anyway.
With that in mind, if you have a "core" library that contains all your math routines, you can simply code two versions: one with a DirectX back-end and one with an OpenGL backend, as long as the interface is the same, you can just switch between them at compile time.
Are you really objecting to the use of DLLs themselves? Last I checked, Ogre3D is LGPL, which means if you link to their libraries statically, then your work is a derived work and you must make your source code public. You do not have to do so if you link dynamically.
The real problem is the use of an abstract base class Renderer that has virtual functions. These can seriously affect performance due to cache misses everytime there is a lookup in the (global) virtual function table.
As I see it, there are three goals. (1) As a developer of LGPL software, you want to give your users the DLL build configurations so they can avoid having to make their code LGPL. (2) You want a graphics library with a platform-independent renderer interface. (3) For performance reasons, you do not want an abstract renderer base class with virtual functions.
Your proposed solution of back ends for DirectX and OpenGL is certainly possible, but there is a price to be paid on the engineering side of things. To illustrate, suppose you have a physics library that accesses data structures in the graphics library in a platform-independent manner. For example, you might want to process the positions in a vertex buffer in order to build a bounding volume for use in collision detection. The access to the positions is through a platform-independent interface. I would really like to have only configurations for "Debug DLL" and "Release DLL", but because the graphics library has "Dx9 Debug DLL", "Dx9 Release DLL", "Wgl Debug DLL", and "Wgl Release DLL", I am forced to have the same four configurations for the physics library (in order that linking succeed). Consequently, I now have PhysicsDx9Debug.dll, PhysicsDx9Release.dll, PhysicsWglDebug.dll, and PhysicsWglRelease.dll, even though the physics library *knows nothing about the underlying graphics API*. Thus, more time is spent in rebuilds and on managing a large number of build configurations. Moreover, the more back ends you add, the configurations grow exponentially.
Essentially, you have a trade-off of performance versus development time. Perhaps the Ogre3D folks opted for less development time, hoping that the performance of applications is acceptable. I made that decision for Wild Magic 4 and earlier, but opted for the non-virtual Renderer in Wild Magic 5, prepared to deal with a large number of build configurations.
In the end, perhaps the real problem is the design of Microsoft Windows DLLs and how creating the *.lib stubs requires all non-virtual functions to be present in the library. Maybe this is possible and I am unaware of it, but it would be nice to have two DLL projects that create A.lib and B.lib that you know are intended to be both linked into an application, but not have missing symbol errors when building A.lib by itself (or B.lib by itself).
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement