Jump to content

  • Log In with Google      Sign In   
  • Create Account


Member Since 19 May 2011
Offline Last Active Nov 17 2013 07:14 AM

Topics I've Started

Shader Library design and implementation

22 May 2013 - 03:35 PM

I'm trying to come up with a good design for a convenient and low-overhead shader library abstracting away low-level graphics API details and allowing me to focus on shaders/game renderer development.


Basically, i'm pursuing the following goals:


1) minimal memory footprint - no string names at run-time, minimal amount of metadata if any;


2) support for hot-reloading modified shader files;


3) support for caching compiled shader bytecode and filtering out duplicates;


How are such shader libraries usually built and how should it be used in actual engine code?

Are there any more performant alternatives to DirectX effects?


The first problem can be solved by storing name hashes instead of full strings (in C++ code compile-time string hashing could be used).

The second one is tough for me, because my shader techniques can depend on global stuff (constant buffers, sampler states and shader resources) and before reloading changed technique i'll have to 'link' it (to avoid recreating those globals).


Right now, my pipeline structure conceptually looks like this:

struct Pipeline
              // global stuff
	Array< RenderTargetResource >		renderTargets;
	Array< DepthStencilStateResource >	depthStencilStates;
	Array< RasterizerStateResource >	rasterizerStates;
	Array< SamplerStateResource >		samplerStates;
	Array< BlendStateResource >		blendStates;
	Array< StateBlock >			stateBlocks;

	Array< CBufferResource >	constantBuffers;       //<= global constant buffers (e.g. cbPerFrame, per-view, per-instance)
             // handles to shader sampler states are stored in the 'samplerStates' array above
	Array< ShaderResource >	shaderResources;   //<= 

	Array< ShaderTechnique >	techniques;

              // all created shader programs (VS/GS/PS) for fixing-up HShaderProgram handles in technique passes (when serialized, they store indices into this array in place of handles)
	Array< HShaderProgram >	programs[ ShaderType::Count ];

	// this is used for locating shader cache on disk
	String		name;

// technique: name and array of passes

// pass - array of shader instances (combinations, identified by a bitmask), default shader instance id,
// and parameter bindings info (cbuffers,samplers,textures) in the form {array, start, count}


Global shader resources are shared between different shader programs, e.g. updating a global constant buffer would look so:


SetStateBlock(Default); // Default is LessEqZNoStencil, NoBlending, CullBack.

CBuffer& perFrame = pipeline.GetCBufferByName("cbPerFrame");

cbPerFrame* data = cast(cbPerFrame*) perFrame.ToPtr();





Managing pipeline dependencies in a submission-based renderer

19 May 2013 - 04:55 AM

  Submission-based engines and stateless renderers (of which the only example i could find is bgfx: https://github.com/bkaradzic/bgfx) are said to be vastly superior than state-machine based APIs. The idea consists in accumulating all the necessary states for making a draw call (helps debugging and filtering out redundant state changes), queuing up the entire frame, sorting the render queue and issuing graphics commands in correct and optimal order.


  But in my engine batched draw calls (RenderItems, RenderOps) are not self-contained, they depend on several global constant buffers (per-frame, per-view, per-object, per-material, per-light, etc.), e.g. the cbPerObject buffer should be updated with fresh data before rendering the actual object and some textures must be updated first before being fed to further stages in the post-processing pipeline and so on.

  Of course, i could create a separate CB for each object, material, etc., but that seems pretty wasteful to me (eschewed approach in DirectX SDK examples).


  How can i ensure the correct execution order (Update,Set,Draw) after sorting the render queue ?




Best way to pass material parameters to shaders

13 May 2013 - 04:01 PM

How do you pass material parameters such as floating-point uniforms to shader programs?


In most engines they query locations of each uniform (by name/semantic) and set them one by one

(e.g. UpdateUniform( int buffer, int offset, const void* data, int size)).


Each uniform, or constant buffer has a shadow copy in sysmem an 'isDirty' flag to skip reuploading unchanged data to videomem.


Wouldn't it be better to have inside the material a parameter blob for just memcpy()ing to entire uniform buffer

(the blob format matches the shader constant buffer layout)? But it won't be possible for modders to assign new shaders to existing materials, right?


Background resource loading - waiting for some item to finish

20 November 2012 - 03:53 AM

Hello !

I'm trying to implement asynchronous resource loading in my engine (by using a separate thread for I/O and decompression) for the first time in my life and i couldn't find solutions to my problem.

1) How should I implement waiting for a specific resource to be loaded?

For instance, when i'm dragging an asset from the asset tree view and dropping it onto the render viewport in my editor,
i need to block the main thread and wait until the resource data is fully loaded or timeout is reached.

2) And how do I specify a timeout value for the load request?

I've read that it's usually done by using request ids/tokens/handles for manipulating individual requests (e.g.: wait/cancel/setPriority).
How is it usually done?

3) How to avoid temporary allocations/extraneous copies during resource loading?

It's bad for performance when the client allocates temporary buffer, issues an async read request into that buffer, instantiates resource, frees the buffer and repeats this procedure, say, a hundred times.

Should i have one streaming buffer which is filled if the user didn't specify her own output buffer?

i've put the whole thing on Pastebin and i'd be very pleased if you could look through the code and spot errors/propose improvements
header file: http://pastebin.com/8pyHnrSh
source file: http://pastebin.com/WFQ1dPyW

Designing efficient Material/Shader system

02 August 2012 - 05:35 AM

I'm having trouble designing a new material system for my engine.

My primary goals are:

0) Minimal CPU overhead and memory footprint

1) I'd like to avoid setting shader parameters (uniforms) one by one
(as in 'shader scripts' approach, 'DirectX effects - style': query location by variable name, need to store metadata about uniforms, etc.).
but that means creating custom structs with the matching layout and 'hardcoding' shaders which is bad (?).

2) Improve efficiency via multithreading (issue graphics API calls in a separate 'render' thread, render commands buffering).
I intend on using two command buffers, there'll be a special UpdateConstantBuffer command,
and i'll need to copy entire constant/uniform buffer contents (each shader parameter) into the command buffer (?).
Seems like a lot of data to memcpy() around...

What are your ideas for designing a powerful, flexible and efficient graphics material system ?

Currently, i have a C++ wrapper class for each shader (generated with a tool, map CB, cast to struct, set params, unmap -> no metadata needed),
and several material classes (Phong, Plain, SSS) which i find cumbersome to use.
i'd rather have one material class, but then i don't know how to set proper shaders,
because material will be generic and shaders are hardcoded.