Why do most tutorials on making an API-agnostic renderer suggest that it's done at buffer/texture level?

Started by
6 comments, last by _the_phantom_ 10 years, 11 months ago

Ive always wondered about this.I wanna port my renderer to OpenGL(currently only using DirectX), however it would seem that having, for instance, a class Mesh that has directx or opengl buffers in it would be way simpler than making a buffer class that either encapsulates an opengl or a d3d buffer, since stuff would get really messy when encapsulating things that differ too much between APIs.For instance in D3D all shaders inherit the same interface and are created and used the same way, while as far as I know in OpenGL shaders required for geometry or tessellation are used a little differently, so making a Shader class that encapsulates all d3d shader functionality might be easy, but making one that can be used for both D3D and GL would be way harder and messier than just making a higher-level RenderEffect class, where you write in it whatever you want for either the d3d or gl code as long as the end result is the same.I'm even considering making an entirely different renderer for OpenGL, it just seems so much easier to do the abstraction on a higher level than the lower one.

>removed<

Advertisement

A few reasons I can think of:

1) it requires (at least superficially) the least amount of thinking / planning to just wrap the low-level API objects

2) by keeping the abstraction at low level you should, theoretically, be able to construct complex API-agnostic higher level rendering code, and not have to duplicate eg. the functionality of Mesh for different APIs

3) A tutorial stays more generally applicable if it doesn't impose its own higher-level constructs

It has potential to get messy, though. For an example here's a list, from my engine, of D3D / OpenGL differences that "leak" from the low level abstraction to the higher. It's not unmanageable, but not pretty either. https://code.google.com/p/urho3d/wiki/APIDifferences

The answer is pretty simple, actually. Most people that write tutorials on the Internet, while certainly well-meaning, have fairly little idea what it is they're doing/talking about. Doubly so for how to explain it. I'll get back to this in a moment.

You're on the right track, though, good to see you've managed to pick the proper level of abstraction.

EDIT: From personal experience, having mesh/particle, light and camera primitives (while exposing some things like shader parameters) seems a good jumping-off point. This is generally flexible enough that you can create most any rendering effect with minimal overhead. I also suggest designing your system to be both very content-oriented and minimally retained; this puts the power in the hands of the artist(s) and also makes debugging/multithreading easier, as all information is usually on-hand.

I don't mean to pick on you, AgentC, but a rebuttal:

A few reasons I can think of:

1) it requires (at least superficially) the least amount of thinking / planning to just wrap the low-level API objects

2) by keeping the abstraction at low level you should, theoretically, be able to construct complex API-agnostic higher level rendering code, and not have to duplicate eg. the functionality of Mesh for different APIs

3) A tutorial stays more generally applicable if it doesn't impose its own higher-level constructs

It has potential to get messy, though. For an example here's a list, from my engine, of D3D / OpenGL differences that "leak" from the low level abstraction to the higher. It's not unmanageable, but not pretty either. https://code.google.com/p/urho3d/wiki/APIDifferences

1) So, in essence, you're making code harder to follow by splitting it up at a very fine level, adding runtime overhead by adding superfluous virtual function calls, etc. just so that you can be typing code into an IDE *right this instant*? That seems a *very* poor tradeoff. This seems to be the thought process behind a lot of tutorials, actually.

EDIT 3: This kind of thinking is also what gave us JavaScript, FWIW.

2) Maybe in theory. By your own admission, though, there are often fundamental differences in how the API works that render this 'abstraction' meaningless anyway-- you still need to add more of them at different levels, which will in turn make code harder to follow.

3) Isn't the point of a tutorial to demonstrate how to take the low-level API and map it to higher-level constructs *anyway*?

clb: At the end of 2012, the positions of jupiter, saturn, mercury, and deimos are aligned so as to cause a denormalized flush-to-zero bug when computing earth's gravitational force, slinging it to the sun.

[..] it just seems so much easier to do the abstraction on a higher level than the lower one.

Whether it's a generic interface to renderer(s) or something else, abstractions tend to work better at a higher level. Otherwise you may as well stick to the low level code minus the abstraction in the first place.

Designing this well is another issue...

[edit] You seem to have come to these conclusions on your own. Perhaps you are just seeking validation.

1) So, in essence, you're making code harder to follow by splitting it up at a very fine level, adding runtime overhead by adding superfluous virtual function calls, etc. just so that you can be typing code into an IDE *right this instant*? That seems a *very* poor tradeoff. This seems to be the thought process behind a lot of tutorials, actually.

EDIT 3: This kind of thinking is also what gave us JavaScript, FWIW.

2) Maybe in theory. By your own admission, though, there are often fundamental differences in how the API works that render this 'abstraction' meaningless anyway-- you still need to add more of them at different levels, which will in turn make code harder to follow.

3) Isn't the point of a tutorial to demonstrate how to take the low-level API and map it to higher-level constructs *anyway*?

Note that I was not trying to advocate a low-level abstraction, but to explain why such approach might be chosen. Like you said, tutorials are many times written without sufficient insight. A *good* low-level abstraction (if one can exist) certainly takes careful planning, and it shouldn't involve virtual functions :)

It has potential to get messy, though. For an example here's a list, from my engine, of D3D / OpenGL differences that "leak" from the low level abstraction to the higher. It's not unmanageable, but not pretty either. https://code.google.com/p/urho3d/wiki/APIDifferences


Thanks. This is a good list of potential problems to look out for.

Learn all about my current projects and watch some of the game development videos that I've made.

Squared Programming Home

New Personal Journal

IMO the portable graphics layer should be:
• Not just a thin wrapper of the underlying API; some abstraction should be done.
• At the very least, the "state machine" behavior of the underlying API must be abstracted away (so that higher level rendering code does not have to worry about 'which states were previously set').
• The majority of API differences should be hidden, and helpers provided for any that do leak (thing like D3D9's idiotic pixel coords, or GL's idiotic NDC are hard to fully hide).
• Keeping the above in mind, it should be as low level as possible so that new rendering code can easily be written across platforms using these abstractions.
• Pet peeve: please don't use abstract base classes (virtual) to separate the interface and implementation(s) of your API. It's unnecessary, it's a huge cause of poor cache behavior and it's a violation of OO design to boot.

I've worked as a graphics-programmer on a game team and on an engine team, so I've been a user of these abstractions and an implementer of them.
When working on the game side, our abstract render layer was hiding about 5 different graphics APIs underneath. I had a lot of unique effects that I had to make for the game, which required low-level graphics programming -- I would much rather do this work once on an abstract API as described above rather than doing it 5 times!

In my API, I expose textures, vertex buffers, index buffers, constant buffers, input layouts (buffer->shader binding), render states and a high-level shader object (vertex/pixel/etc all set in one go) because then you can write nearly any graphical effect on top of this API, instead of repeating work.
As long as you abstract away the state machine and submission of commands, it's fairly easy to hide most underlying API differences.

It kinda depends on who the target users of your abstract API are. If you don't want/need the users to be able to implement new lighting pipelines, shadowing techniques or special effects, then you can make your API very high level without any worries. You then implement this high level API once for each 'back-end' API, with a large amount of very different code. A graphics programmer maintains the back-end.
If however, you're making an engine that should be able to be used by graphics programmer to implement new features, then your API needs to be a bit lower level, as described above. You have graphics-programmers maintaining the fairly thin/simple back-ends (not any real graphical techniques here, just mapping one abstraction to another) and you also have graphics-programmers implementing graphical techniques on top of your API.

I actually recommend a 3-tier system, where I've described the lowest tier above (the abstraction of D3D/GL/etc into a single stateless API), then there's a middle tier that implements lighting/shadows/effects/etc on top of this API and exposes high level objects like lights and models, which are used by the top tier to build a game without specialist graphics knowledge.

In my API, I expose textures, vertex buffers, index buffers, constant buffers, input layouts (buffer->shader binding), render states and a high-level shader object (vertex/pixel/etc all set in one go) because then you can write nearly any graphical effect on top of this API, instead of repeating work.

We do much the same, although instead of having vb, ib, cb etc types everything is unified under a 'buffer' type where usage/construction sets up the underlying buffer type. We also don't differentiate between 'textures' and 'render targets'; all are unified under a 'surface' type. The logic being the people creating the buffers/surfaces already know their intent and many of the operations are the same anyway. Data streaming off disk has it's type information embedded in the header section which can be directly de-serialised for the create functions when loading.

This topic is closed to new replies.

Advertisement