Thoughts on managing resources (design)?

Started by
6 comments, last by _the_phantom_ 7 years, 4 months ago

Hi all,

Last few weeks I've made nice progress in creating the base for my new (d3d11 based) engine/ framework.

With the positive remark; also thanks to all the support and help from the community (both forum and chat). So thanks! :cool:

Where possible I try to expose as less 'low tech' stuff/ implementation details from my low-level D3D11 renderer.

For example, the user can easily add, use, update, release a rendertarget or a constant buffer for example.

Now my question regards the positioning of resources, and potentially the creation of a resource manager (system).

I've thought about creating a resource manager to manage assets: textures, models, sounds, music, texts etc.

Where I do reference counting/ prevent loading the same resources again, when switching a scene/ level. Scenes/ levels will be managed by the SceneManager.

So far all good, well, at least a clear direction to follow.

But my question is, where would you/I position shaders/ my ShaderManager.

I don't believe it should be part of the low-level renderer (which I wanted to do first, so design below). I also don't think it shouldn't be 'part'/ member of the scene manager.

Because shaders are not perse scene/ level bound.

In short; where would you 'place' Shaders/ the ShaderManager?

My current thought is that the approach in the design below, will be relatively easily usable, with only the 'con' that the shaders will be stored within the low-level renderer system

(maybe have the manager in the low-level renderer, and simply use pointers to the actual shaders, stored in the resource manager/system?)

I have the same question for managing renderstates, which I believe will be linked to materials or specific rendering conditions.

Any thoughts are appreciated.

crealysm11_design_v0.5.jpg

Crealysm game & engine development: http://www.crealysm.com

Looking for a passionate, disciplined and structured producer? PM me

Advertisement
I'm by no means an expert on this stuff specifically, struggle with it somewhat myself, and have yet to see a system that doesn't suck in some way or another here on any engine AAA or hobby that I've worked with, but my current thinking:

Separate the high-level and low-level concept of resources. There's a huge difference between "game object Mean_Orc requires the meanorc.mesh resource" and "the collection of buffers, states, srvs, and other device resources that represent meanorc.mesh to the renderer and GPU device."

A high-level resource object might be owned by a high-level resource manager or resource factory, e.g. a generic MeshResourceFactory that knows how to load .mesh files and hand out Resource<Mesh> handles and so on. The low-level renderer is then notified of the new resource and it creates and initializes all the buffers and whatnot, and possibly also does the disk load into those buffers so you aren't keeping file contents in memory unnecessarily.

This has a few advantages IMO:

- High-level game systems that need to know about resource graphs, dependencies, etc. don't take a hard dependency on the renderer, e.g. a headless tool can create and manipulate Resource<Mesh> objects without ever instantiating an instance of D3D11Renderer.
- Device resets or other invasive changes are non-invasive to game code.
- Streaming is simpler because there's already a clean separation between high-level records of in-use resources and low-level records of MIPs/LODs/etc. loaded into the GPU.
- Core rendering algorithms can work directly with GPU resources and avoid overhead or bloat requires for gameplay tracking, like game object reference counting, resource sets, or various other metadata.
- It makes the code structure and portability a little more obvious since you can cleanly separate your game resources from your GPU resources.

Again, though, I'm by no means a graphics guru and this is all just my current opinion based on hobby experiments and unshipped playground code, not on how we do it For Realz at the day job.

Sean Middleditch – Game Systems Engineer – Join my team!

Thanks, my take from this is that I should/could (and will :)) apply the following guiding principles:

- don't mix low-level and high-level in 1 system/ class (and hide low-level implementation from the caller)

- separate resource storing/memory and actual using/ processing

What I'm not sure about, is if I should handle SRV's and shaders as resources in this kind.

If so, I could have the shader manager 'in' the low-level renderer, and let it get/ give out pointers to resources in the resource manager (in this case shaders). For assets it's clear, the scene manager will then communicate with the resource manager.

Personally I believe I shouldn't overthink it too much and just keep/ have the SRV's and shadermanager (with the shaders), in the low-level renderer. So the caller/user can simply say, set the shader to the shader with ID x / 123 (everything else will be handled behind the scenes). Where the x or 123 will be defined in a material or other conditions. The same goes for renderstates.

All assets (sounds, music, text, meshes, textures etc.) will then follow the principle above: actual memory/ storage within the resource manager. And the user can retrieve a pointer or handle to the resource (and if not yet loaded, load it at that moment etc.).

Any suggestions on this approach?

Crealysm game & engine development: http://www.crealysm.com

Looking for a passionate, disciplined and structured producer? PM me

i think i'd have to second Sean's opinion.

There seems to be a distinct advantage to using asset pools which are then referenced by things like entity lists. then the only coupling between the two is an ID number. This leaves the asset pool free to handle low level stuff, and the entity itself doesn't have to worry about it.

i've also found that games can often be divvied up into the following modules:

program (init_program, run main_menu, end_program)

game (init_game, run_game [IE: main game loop], end_game)

render (uses entity lists, assets pools, and graphics engine to render the scene)

input (get and process input, or save for deferred processing later)

update (move/run everything!)

graphics engine (draws stuff)

audio library (plays wavs)

entity lists (targets, projectiles, dropped objects, temporary particle emitters, etc)

asset pools (meshes, textures, audio files, etc)

low level modules (timers, math libraries, etc)

Norm Barrows

Rockland Software Productions

"Building PC games since 1989"

rocklandsoftware.net

PLAY CAVEMAN NOW!

http://rocklandsoftware.net/beta.php

keep/ have the SRV's and shadermanager (with the shaders), in the low-level renderer. So the caller/user can simply say, set the shader to the shader with ID x / 123

if it smells like an asset...

since a shader is an asset that can be specified for use in a drawing call, odds are you should treat it like any other asset, such as a mesh or texture. while its code instead of data, its still a "demand-load-paged asset" type of thing in the way its used.

Norm Barrows

Rockland Software Productions

"Building PC games since 1989"

rocklandsoftware.net

PLAY CAVEMAN NOW!

http://rocklandsoftware.net/beta.php

Shaders occupy an interesting point in the asset chain; they are indeed assets but they are low level ones.

Much like meshes, sub-mehes, vertex and index buffers there is an argument they shouldn't be directly touched by client code aka the higher level (a counter argument could be made for mesh/sub-mesh for dynamic things, but I would perhaps structure that in another way.)

Ultimately a shader is something which would be tied to a material to give that material a property - in a company or group setup there would be a couple of people involved in this; a tech artists and potentially a graphics programmer would define the shaders, artists however would apply materials to models without ever touching the shaders directly (they would see slots for textures and values they can tweak).

In my mind, while shaders are indeed an asset, they are a low level asset, generally ones which at a higher level won't be referenced on their own, instead are implicitly handled via materials which are in turn applied to submeshes of meshes of models. (Again, at the higher level you don't care about the submesh/mesh stuff, you simply try to load a model and the rest 'comes with it' as it were).
Thanks. So basically the low level resource is requested by the Material (not low-level). And the user sets just the material.

Same for renderstates, which can also have other "requesters" then a material.

Crealysm game & engine development: http://www.crealysm.com

Looking for a passionate, disciplined and structured producer? PM me

Generally the material would set render states (part of the pre-draw sorting system so its kinda needed).
Post processing would be the other system, but that's technically just another material system, all be it one which does full screen and/or compute based passes.

This topic is closed to new replies.

Advertisement