Jump to content
  • Advertisement
Sign in to follow this  
Anfaenger

Data-Driven Renderer - how to modify the pipeline on-the-fly?

This topic is 672 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi!

 

I'm creating a data-driven rendering engine which is loosely based on Bitsquid/Stingray slides.

Rendering pipelines are compiled from descriptions written in a JSON-like DSL.

Until now, everything was nice and clean, the system allows to quickly prototype new renderers without writing tons of C++ glue.

 

But things get messy when the rendering pipeline must be changed at run-time (e.g. the player wants to toggle a certain feature on/off).

 

For instance, in the pipeline description all render targets are fixed, but if the user enables SSAO, the ueber_postprocessing shader must now output to two render targets instead of one. Similarly, if FXAA is disabled, the postprocessing shader must write straight to the backbuffer. There's a bunch of lesser problems (e.g. changing the render target clear colors to white when I want to take screenshots for whitepapers).

 

Here's my actual rendering engine pipeline description (with ugly HACKs):

|	Tiled deferred rendering pipeline description.
|

; dependencies
includes = [
	'renderer_common'    ; defines common render states
	'deferred_common'    ; defines common render targets and the main depth-stencil
]

color_targets = [
	; R8G8B8 - world-space normal, A8 - specular intensity
	{
		name = 'GBufferTexture0'
		depends_on = 'BackBuffer'
		;ScaleX	=	1.0
		;ScaleY	=	1.0
		sizeX = { size = 1 relative = 1 }
		sizeY = { size = 1 relative = 1 }
		format	=	'RGBA8'
		auto_clear = 1
		;Info	=	'Normals & Specular intensity'
	}
	; R8G8B8 - albedo, A8 - specular power
	{
		name = 'GBufferTexture1'
		depends_on = 'BackBuffer'
		;ScaleX	=	1.0
		;ScaleY	=	1.0
		sizeX = { size = 1 relative = 1 }
		sizeY = { size = 1 relative = 1 }
		format	=	'RGBA8'
		auto_clear = 1
		;Info	=	'Diffuse & Specular power'
	}
]

depth_targets = [
	; CSM/PSSM
	{
		name = 'ShadowMap'
		sizeX = { size = 2048 relative = 0 }
		sizeY = { size = 2048 relative = 0 }
		format	=	'D32'
		sample = 1
	}
]

|
35. Layers Configuration:
    Defines the draw order of the visible batches in a game world 
    Layers are processed in the order they are declared 
    Shader system points out which layer to render in 
|
layers =
[
; HACK:
	{
		id = 'FrameStart'
		render_targets = [
			{ name = '$Default' clear = 1 }
		]
		depth_stencil = { name = 'MainDepthStencil' clear = 1 }
		profiling_scope = 'Clear FrameBuffer'
	}

	; Directional light
	{
		id = 'ShadowMap0'
		; depth-only rendering
		depth_stencil = { name = 'ShadowMap' clear = 1 }
		sort = 'FrontToBack'
		profiling_scope = 'ShadowMap0'
	}
	{
		id = 'ShadowMap1'
		; depth-only rendering
		depth_stencil = { name = 'ShadowMap' }
		sort = 'FrontToBack'
		profiling_scope = 'ShadowMap1'
	}
	{
		id = 'ShadowMap2'
		; depth-only rendering
		depth_stencil = { name = 'ShadowMap' }
		sort = 'FrontToBack'
		profiling_scope = 'ShadowMap2'
	}
	{
		id = 'ShadowMap3'
		; depth-only rendering
		depth_stencil = { name = 'ShadowMap' }
		sort = 'FrontToBack'
		profiling_scope = 'ShadowMap3'
	}


	; G-Buffer Stage: Render all solid objects to a very sparse G-Buffer
	{
		id = 'FillGBuffer'
		; Clear Geometry buffer.
		; Bind render targets and main depth-stencil surface.
		render_targets = [
			{ name = 'GBufferTexture0' clear = 1 }	; normals
			{ name = 'GBufferTexture1' clear = 1 }	; albedo
		]
		depth_stencil = { name = 'MainDepthStencil' clear = 1 }
		;render_state = 'Default'
		;default_shader = ?
		;filter = Solid/Opaque only
		sort = 'FrontToBack'	; Draw opaque objects front to back
		profiling_scope = 'Fill G-Buffer'
		; Fill buffers with material properties of *opaque* primitives.
	}
	; G-buffer has been filled with data.


; HACK:
{
	id = 'ClearLightingTarget'
	render_targets = [
		{ name = 'LightingTarget' clear = 1 }
	]
}


	; point lights -> 'LightingTarget'
	{
		id = 'TiledDeferred_CS'
		; Bind null color buffer and depth-stencil.
		; No need to clear, we write all pixels.
		profiling_scope = 'TiledDeferred_CS'
	}
	; Render deferred local lights.
	{
		id = 'DeferredLocalLights'
		render_targets = [
			{ name = 'LightingTarget' }
		]
		depth_stencil = { name = 'MainDepthStencil' }
		profiling_scope = 'Deferred Local Lights'
	}
	; Apply deferred directional lights.
	{
		id = 'DeferredGlobalLights'
		render_targets = [
			{ name = 'LightingTarget' }
		]
		; Cannot bind the main depth-stencil surface - we read from it.
		profiling_scope = 'Deferred Global Lights'
	}

	; Forward stage
	{
		id = 'Unlit'
		render_targets = [
			{ name = 'LightingTarget' }
		]
		depth_stencil = { name = 'MainDepthStencil' }
		profiling_scope = 'Unlit'
	}	

	{
		id = 'SkyLast'
		render_targets = [
			{ name = 'LightingTarget' }
		]
		depth_stencil = { name = 'MainDepthStencil' }
		profiling_scope = 'Sky'
	}

	; Alpha-Blended (Order-Dependent) Transparency
	{
		id = 'Translucent'
		render_targets = [
			{ name = 'LightingTarget' }
		]
		depth_stencil = { name = 'MainDepthStencil' }
	}

	; Blended Order-Independent Transparency (OIT):
	;
	{
		id = 'BlendedOIT'
		render_targets = [
			{ name = 'BlendedOIT_Accumulation' clear = 1 }
			{ name = 'BlendedOIT_Revealage' clear = 1 }
		]
		; Keep the depth buffer that was rendered for opaque surfaces
		; bound for depth testing when rendering transparent surfaces,
		; but do not write to it.
		depth_stencil = { name = 'MainDepthStencil' }
	}
	
	; Combine: LightingTarget -> FrameBuffer
	; Blended OIT: 2D Compositing Pass
	{
		id = 'DeferredCompositeLit'
		render_targets = [
;TEMP HACK:
;			{ name = '$Default' }
			{ name = 'FXAA_Input' }
		]
		profiling_scope = 'Tonemap'
	}


	; thick wireframes should be antialiased
	{
		id = 'DebugWireframe'
		render_targets = [
;TEMP HACK:
;			{ name = '$Default' }
			{ name = 'FXAA_Input' }
		]
		depth_stencil = { name = 'MainDepthStencil' }
		profiling_scope = 'DebugWireframe'
	}

	
	; Fast Approximate Anti-Aliasing (FXAA)
	{
		id = 'FXAA'
		render_targets = [
			{ name = '$Default' }
		]
	}

	; Draw debug lines and GUI above everything else
	{
		id = 'DebugLines'
		render_targets = [
			{ name = '$Default' }
		]
		depth_stencil = { name = 'MainDepthStencil' }
		profiling_scope = 'DebugLines'
	}
	{
		id = 'DebugScreenQuads'
		render_targets = [
			{ name = '$Default' }
		]
		profiling_scope = 'DebugScreenQuads'
	}

	; draw GUI above everything else
	{
		id = 'GUI'
		render_targets = [
			{ name = '$Default' }
		]
		profiling_scope = 'ImGui'
	}
]

1) What is the cleanest way to allow on-the-fly modification of the rendering pipeline in a data-driven graphics engine?

2) How should render targets be described in a flexible rendering pipeline? How to specify that a render target should be taken from a render target pool?

Edited by Anfaenger

Share this post


Link to post
Share on other sites
Advertisement

1) What is the cleanest way to allow on-the-fly modification of the rendering pipeline in a data-driven graphics engine?
 Reload everything?

 

 2) How should render targets be described in a flexible rendering pipeline? How to specify that a render target should be taken from a render target pool?
I put a "shared" flag in each render target. If they're shared, they go to a name->target map. All targets first look into that map to see if there is a target with the same id, or create it otherwise. If a target is created and it has the shared flag, I add it to the pool, otherwise I just leave it alone in the render pass.

Share this post


Link to post
Share on other sites
* Make it code driven :lol:
One non-facetious way of doing that would be to use something like Lua to store the graph - it's good at storing JSON-like data tables, but is also a fully fledged language.
* Add a small amount of branching logic to your DSL - e.g. this branch of the graph only active when blah condition is true.
* Make a whole boatload of different data files and load the right permutation depending on the situation.


As for the pooling, we describe every target as if it's unique to its pass - e.g. Bloom declares that it requires a half-res buffer, then DOF also declares that it requires a half-res buffer. If you then walk through the graph, allocating a target as a pass needs it and returning it to a pool when the pass is done (using only size/format, not name), then you've now got a minimal pool of targets that can be reused across passes (e.g. DOF and bloom will reuse the same half-res buffer).

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!