A dynamic material system (and shaders)
Author
I've taken a very strong liking to the surface shader concept detailed in these slides: http://developer.amd.com/assets/Andersson-Tatarchuk-FrostbiteRenderingArchitecture(GDC07_AMD_Session).pdf (starting at page 23)
I'm trying to develop an API independent material solution for my rendering framework. The problem I find with the traditional usage of the programmable pipeline is you need a very large pool of shaders (e.g. for every possible lighting combination, blend effects, etc) which is quite a tedious job to create and maintain those effects.
I came up with the idea of procedurally creating the shaders at run-time (and when researching that I cam across another thread on Gamedev about JIT shaders) but it still left the problem of how you were suppose to custom shaders with the JIT solution for special cases (the most common example being water). That's when I came across the Frostbite slides.
The surface shader has caught on ever since, though it seems more like pixel shader solution when I'm wanting a solution to handle my entire material pipeline.
I'm thinking materials should have 2 interconnected graphs, a per-vertex graph and a per-pixel graph that are connected together. I think it would be much more elegant to have only a single graph (have my framework determine what can be done per-vertex or per-pixel), and to add support for geometry shaders and any future shaders would be add a few new classes of nodes.
A quick scribble of how the first idea of interconnected graphs would kind of look/connect together:
http://img210.imageshack.us/my.php?image=graphexampleyk7.png
Every node will have an underlying HLSL and GLSL (or perhaps CG to reduce the need for redundant code) making it easy to add new nodes (or complex nodes to avoid spaghetti graphs) or view the code behind nodes to see how they work. Having HLSL/GLSL/CG code however, can be viewed as a negative thing if I wish to port to an different API in the future or to an embedded system which uses a fixed function pipeline.
Lights (spot light, point light, directional light) will be their own independent graph (using the same system as creating materials) that is snapped in at run-time when they're added to the scene. So you can add any new types of lights easily.
Lighting can be added into the material's graph as a placeholder node (perhaps there is a need a per-light section of the material's graph?). It's hard to think up a solution for this because there are special properties (e.g. specular) that rely on both the properties of the light and the material, and how are you suppose to connect the inputs when you're not sure. This place holder will be filled in at run-time depending on the lights affecting the object.
There are even scene specific things, perhaps an environmental graph that needs to fit in somehow (e.g. fog).
So what are the general thoughts and improvements to this system?
[Edited by - MessiahAndrw on December 28, 2008 4:41:23 AM]
Quote:The problem I find with the traditional usage of the programmable pipeline is you need a very large pool of shaders (e.g. for every possible lighting combination, blend effects, etc) which is quite a tedious job to create and maintain those effects.
This is the wrong assumption. Your pool of shaders is very small because you do not want to switch shaders very often -> quite expensive.
If you start with the assumption that you only need up to 20 different materials which is a sane number, the number of shaders you really use is quite small.
So all the procedural shader stuff is not necessary. It wouldn't allow you to optimize shaders in any meaningful way.
A premise to my post: I don't like when somebody asks for feedback/help and people answer "don't do that". I think everybody's free to implement/create/use whatever they want. If you think people are wrong you are free to tell them, but to keep saying they took a wrong path is useless and arrogant.
Another premise: at this time I'm adding (actually trying to) more variety to the scenes rendered by my code. Of course this is an area where a tool like the one you want to develop could express itself at its best. Despite that I'm one of the guys believing such a system isn't a viable solution to solve my problems.
There are some professionals (which are much more competent than me) that shared their point of view about this subject. I suggest you to have a look at their blogs:
http://realtimecollisiondetection.net/blog/?p=73
http://diaryofagraphicsprogrammer.blogspot.com/2008/09/shader-workflow-why-shader-generators.html
As for me, I'd like to share some general thoughts. I believe a shader generator is a cumbersome solution to a non-problem.
The non-problem is generally referred to as "shader explosion" (i.e. the overwhelming amount of shader permutations needed).
If you take a standard rendering pipeline as a reference, the number of different shaders can grow to a limit which makes their mantainance/optimization a nightmare. When you combine several rendering techniques with all possible combinations of lights (number and type) that can affect a mesh, the result is a very very large shader array. Back in the days where vertex and pixel shaders were 100% asm this was the case.
As of 2008, nobody is going to use a standard rendering pipeline, since many newer (and better) pipelines emerged (multipass/deferred/light prepass/etc) to limit this issue. If your application needs 200+ shaders, it doesn't matter how do you generate them, there's surely something wrong in your rendering pipeline. Just like the fastest primitive is the one you don't draw, the easier shader to write is the one which you don't write at all!
IMHO any modern rendering pipeline will make shader explosion a non-problem.
As I said, I'm doing some research to differentiate my scene materials as much as possible. I'm not limited by the amount of shaders, I'm limited by the time/money I want/can invest on writing new shaders.
At a first glance you'd expect a shader generator could help me. I hope so, but I doubt it will make a difference.
Let's see which is the real point. The problem I'm facing is this: "which level of material differentiation do I need? how do I expect to reach my goal?".
I can see two main areas to improve the rendering quality:
- parameters (this includes any shader input, including textures)
- rendering techniques
As for the parameters, think about a shader as a box with plugs. You have a simple shader which draws a texture mapped mesh. The texture map is obviously an input. There's no reason to generate one shader for each texture map, as the shader code isn't going to change. The same applies to common parameters. If your shader performs a complex operation which needs to be tweaked I guess you'll not use magic numbers (i.e. nobody is so insane to generate a new shader for every different specular level value!).
To sum up I have a base rendering technique and I would like it to perform an operation whose results depend upon a set of parameters which are set by a "material class". Here's the first problem. How can a shader generator help me? Part of the data needed by a shader is inside a material class, as I want to reuse the same shader as much as possible. In order to assign a generated shader to a material, I have to be sure the material can hold everything needed to tweak the shader. It turns out your shader generator should also take care of generating a proper material class.
Suggestion number one: "after the shader has been generated, also generate a "template material"/"material descriptor" for it. According to the material descriptor, dynamically add parameters to your material. You need to add support for dynamic parameters in your base material class (and associate them with shader parameters), otherwise your ability to tweak the shaders will be limited by your material class". Sorry but that's the best I can think of ATM.
Problem number two: changing rendering technique. Assuming you really need 20-30 rendering techniques, some of them can require special inputs. The rendering pipeline should be aware of the fact a particular rendering technique is in use/supported. Even a simple shadowmap needs to be generated somewhere in the pipeline and then sent to your shader as an input. How do you model that? Your problem here isn't about the graph itself. The problem is you need to provide a meaningful input to it.
Suggestion number two: "split the shader/graph inputs into static and dynamic. a static input is just a texture map that is assumed to be available. a dynamic input is a resource which *could* be generated by another piece of the pipeline. When compiling/generating your graph, the inputs are "validated". Validation process should ensure your input is always meaningful, despite the resource you need could not be available(!!!!). If this sounds weird, follow me. You have 200 lights, they should cast shadows but you don't have enough cycles to regenerate 200 shadow maps every frame. There's a part of the pipeline caching shadowmaps. Only 10 lights out of 200 actually have a shadow map. What about the others? Here's when the validation process comes into play. I can think of two possible solutions. The first one being to require the artist to include a default alternative input (say a 1x1 float_max texture for a shadowmap), the second one being the ability to compile all possible shader combinations when one (or more) dynamic input isn't available. If the graph is properly built, you should be able to exclude nodes (replacing them with a default value set as a "node property"). This way you are effectively OPTIMIZING your shaders, since your material class can pick the one which performs only the operations needed for the inputs he got. Hopefully the state changes will be balanced by the faster shader code. You could pick up the proper shader using a map with a bitfield as key. 32 parameters per shader should be enough. What's left is how to generate that input textures. You probably need to indentify where in the pipeline you should generate a texture, I'd suggest to create a class representing a "pass" (in a sense similar to a depth prepass). You should also implement a base cacher class and figure out a way to bind a N shader outputs to M shader inputs (M >= N)." It doesn't sound something impossible to do, but surely it's a lot of work. Again, sorry but this is the best I can think of.
If you ask me, the amount of work required to develop such a pipeline and a good tool is too much if we are only talking about 20 shaders. I don't know about 2 years from now, but as of today, I don't plan to develop a shader generator.
Another premise: at this time I'm adding (actually trying to) more variety to the scenes rendered by my code. Of course this is an area where a tool like the one you want to develop could express itself at its best. Despite that I'm one of the guys believing such a system isn't a viable solution to solve my problems.
There are some professionals (which are much more competent than me) that shared their point of view about this subject. I suggest you to have a look at their blogs:
http://realtimecollisiondetection.net/blog/?p=73
http://diaryofagraphicsprogrammer.blogspot.com/2008/09/shader-workflow-why-shader-generators.html
As for me, I'd like to share some general thoughts. I believe a shader generator is a cumbersome solution to a non-problem.
The non-problem is generally referred to as "shader explosion" (i.e. the overwhelming amount of shader permutations needed).
If you take a standard rendering pipeline as a reference, the number of different shaders can grow to a limit which makes their mantainance/optimization a nightmare. When you combine several rendering techniques with all possible combinations of lights (number and type) that can affect a mesh, the result is a very very large shader array. Back in the days where vertex and pixel shaders were 100% asm this was the case.
As of 2008, nobody is going to use a standard rendering pipeline, since many newer (and better) pipelines emerged (multipass/deferred/light prepass/etc) to limit this issue. If your application needs 200+ shaders, it doesn't matter how do you generate them, there's surely something wrong in your rendering pipeline. Just like the fastest primitive is the one you don't draw, the easier shader to write is the one which you don't write at all!
IMHO any modern rendering pipeline will make shader explosion a non-problem.
As I said, I'm doing some research to differentiate my scene materials as much as possible. I'm not limited by the amount of shaders, I'm limited by the time/money I want/can invest on writing new shaders.
At a first glance you'd expect a shader generator could help me. I hope so, but I doubt it will make a difference.
Let's see which is the real point. The problem I'm facing is this: "which level of material differentiation do I need? how do I expect to reach my goal?".
I can see two main areas to improve the rendering quality:
- parameters (this includes any shader input, including textures)
- rendering techniques
As for the parameters, think about a shader as a box with plugs. You have a simple shader which draws a texture mapped mesh. The texture map is obviously an input. There's no reason to generate one shader for each texture map, as the shader code isn't going to change. The same applies to common parameters. If your shader performs a complex operation which needs to be tweaked I guess you'll not use magic numbers (i.e. nobody is so insane to generate a new shader for every different specular level value!).
To sum up I have a base rendering technique and I would like it to perform an operation whose results depend upon a set of parameters which are set by a "material class". Here's the first problem. How can a shader generator help me? Part of the data needed by a shader is inside a material class, as I want to reuse the same shader as much as possible. In order to assign a generated shader to a material, I have to be sure the material can hold everything needed to tweak the shader. It turns out your shader generator should also take care of generating a proper material class.
Suggestion number one: "after the shader has been generated, also generate a "template material"/"material descriptor" for it. According to the material descriptor, dynamically add parameters to your material. You need to add support for dynamic parameters in your base material class (and associate them with shader parameters), otherwise your ability to tweak the shaders will be limited by your material class". Sorry but that's the best I can think of ATM.
Problem number two: changing rendering technique. Assuming you really need 20-30 rendering techniques, some of them can require special inputs. The rendering pipeline should be aware of the fact a particular rendering technique is in use/supported. Even a simple shadowmap needs to be generated somewhere in the pipeline and then sent to your shader as an input. How do you model that? Your problem here isn't about the graph itself. The problem is you need to provide a meaningful input to it.
Suggestion number two: "split the shader/graph inputs into static and dynamic. a static input is just a texture map that is assumed to be available. a dynamic input is a resource which *could* be generated by another piece of the pipeline. When compiling/generating your graph, the inputs are "validated". Validation process should ensure your input is always meaningful, despite the resource you need could not be available(!!!!). If this sounds weird, follow me. You have 200 lights, they should cast shadows but you don't have enough cycles to regenerate 200 shadow maps every frame. There's a part of the pipeline caching shadowmaps. Only 10 lights out of 200 actually have a shadow map. What about the others? Here's when the validation process comes into play. I can think of two possible solutions. The first one being to require the artist to include a default alternative input (say a 1x1 float_max texture for a shadowmap), the second one being the ability to compile all possible shader combinations when one (or more) dynamic input isn't available. If the graph is properly built, you should be able to exclude nodes (replacing them with a default value set as a "node property"). This way you are effectively OPTIMIZING your shaders, since your material class can pick the one which performs only the operations needed for the inputs he got. Hopefully the state changes will be balanced by the faster shader code. You could pick up the proper shader using a map with a bitfield as key. 32 parameters per shader should be enough. What's left is how to generate that input textures. You probably need to indentify where in the pipeline you should generate a texture, I'd suggest to create a class representing a "pass" (in a sense similar to a depth prepass). You should also implement a base cacher class and figure out a way to bind a N shader outputs to M shader inputs (M >= N)." It doesn't sound something impossible to do, but surely it's a lot of work. Again, sorry but this is the best I can think of.
If you ask me, the amount of work required to develop such a pipeline and a good tool is too much if we are only talking about 20 shaders. I don't know about 2 years from now, but as of today, I don't plan to develop a shader generator.
Quote:to keep saying they took a wrong path is useless and arrogant.to make up for my arrogance, here are a few ideas on how to design a system like this.
1. It depends on your game. Whatever your game is about will define which kind of materials you will use. Whatever materials you use you want to implement different combinations of shaders. So I would ask myself first this question.
2. It depends on how you implement lights. You won't achieve many fully dynamic lights with a traditional renderer design. If you go with a deferred lighting model, you will mainly have three shaders for each of the light types and then probably a few one for things like shadows, postfx, reflections etc. and maybe a few changes to the main three shaders for the restricted amount of materials that a deferred lighting scheme can handle on DX9 hardware (if you have really powerful hardware you might think about an index value in the G-Buffer that references different shaders and switches per-screenspace pixel the material. With a Light Pre-Pass you will have your three light shaders for the pre-pass and then a variety of shaders for the main rendering path. The variety is restricted on the light properties you store in the Light Buffer. If you have really powerful hardware you can store an index and switch shaders per-screenspace pixel ....
3. You might also think about the granularity of such a system. If you implement Oren-Nayar, Ashikhmin-Shirley, Cook-Torrance and other lighting models you want to think about how to combine those. Will each of those lighting models represent diffuse or specular or will there be a finer granularity?
4. You might also think about different normal data quality levels, like tangent space normals, derivative normal maps, height maps for normal blending and how you let artists pick one of those while at the same time making sure that the source asset is really in place.
5. For PostFX, Shadows, reflections and other separable high-level graphics like a dynamic sky system you do not want to generate those shaders because you only need one for each of those and optimizing those is a tough task.
Quote:Original post by wolfQuote:to keep saying they took a wrong path is useless and arrogant.to make up for my arrogance, here are a few ideas on how to design a system like this.
I wasn't referring to your reply as I noticed it only after clicking "post reply" button. I'm talking about countless threads when people ask about developing an engine and get 20 replies saying only "write games, not engines" plus a bare link. This thread risks to become nothing but a long list of "don't do it", without further explanations. :)
It's probably OT but this grabs my attention:
Quote:
if you have really powerful hardware you might think about an index value in the G-Buffer that references different shaders and switches per-screenspace pixel the material.
IIRC in Stalker they use Material IDs to index a dot(n,l) texture. If you are referring to this value, in case of different shaders you are forced to run multiple passes hoping dynamic branches (and locality) will help you. That sounds like a performance killer, as no stenciling/zcull can help you discard pixels before executing the pixel shader, because the reference value is stored in the gbuffer. Do you expect an ubershader with huge switches to perform better or do you think a multipass approach would be desirable?
I've a mechanism like this (although ATM I don't perform any additional pass), it's working but I'd like to design it properly.. and.. well.. I'm stuck because I would like to find a simple solution but I'm unable to do it.
Every solution I can think of requires an huge amount of work to provide an extendable and robust material ID system. My problem with material IDs is once developed they turn into an evil beast having a lot more to do with craftsmanship than engineering.
Have you ever implemented a fully functional Material ID system?
Quote:IIRC in Stalker they use Material IDs to index a dot(n,l) texture. If you are referring to this value, in case of different shaders you are forced to run multiple passes hoping dynamic branches (and locality) will help you. That sounds like a performance killer, as no stenciling/zcull can help you discard pixels before executing the pixel shader, because the reference value is stored in the gbuffer. Do you expect an ubershader with huge switches to perform better or do you think a multipass approach would be desirable?latest hardware seems to be able to do this fast enough ... I haven't implemented it but a friend at NVIDIA did ... obviously I prefer the Light Pre-Pass renderer design that does not offer this challenge.
I wouldn't be so sure about Stalker. The article was written three years before the game shipped. I would expect a lot of changes in the meantime ... I think some of the people that wrote the article left the team also. The idea to build a material system with a 3D texture does not sound reasonable to me :-)
With G80 and newer hardware, this seems to be an option. I assume it is the same for ATI hardware because they had fast branches since a while.
You would have to try out implementing 2D texture arrays and this and see what is faster. Because you will need the 2D texture support anyway on lower end hardware you won't loose much time.
Quote:Original post by wolf
latest hardware seems to be able to do this fast enough ... I haven't implemented it but a friend at NVIDIA did ... obviously I prefer the Light Pre-Pass renderer design that does not offer this challenge.
I can't wait to read the part about the Light Pre-Pass renderer in your D3D10 book.. ;)
Quote:
The idea to build a material system with a 3D texture does not sound reasonable to me :-)
With G80 and newer hardware, this seems to be an option. I assume it is the same for ATI hardware because they had fast branches since a while.
You would have to try out implementing 2D texture arrays and this and see what is faster. Because you will need the 2D texture support anyway on lower end hardware you won't loose much time.
As for texture arrays, reference API is still DX9. :(
Anyhow I abandoned the idea of using multipasses, as the amount of draw calls would become unsustainable.
Author
I appreciate all of your feedback.
I understand where you're coming from if you prefer other solutions rather than procedurally generate your shaders, but this is my framework which is all about experimentation (and I don't have a deadline so I'm able to invest time).
Right now it's a trade off between flexibility vs performance vs artist-friendly.
In a perfect scenario, I would rather not have any hard coded hlsl/glsl/cg in the graph to make it fully API independent (and emulatable on a fixed-function system). I see complex algorithms leading to spaghetti graphs (an entire graph could be encapsulated inside a single node - much like a function call). The main issue I see with this system though is performance wise from the code generation (no hand coded optimisations). It is artist friendly (to technical artists) without having to write in a shader language, but with enough "function nodes" you could abstract away the complexity.
The most performance-ideal version of the system is to have huge do-it-all nodes hardcoded in CG. e.g. a node called DiffuseAmbientSpecularReflectionNormalMap that takes a Diffuse colour value, an Ambient colour value, specular values, a reflection value, a normal, which you can either connect a texture sampler to or a hard float. This is also the most artist friendly since it's also simple to understand.
The most artist friendly version would be to imitate Maya's or Max's material system. But I have seen some really huge materials made with this that it'd be a performance issue.
I understand where you're coming from if you prefer other solutions rather than procedurally generate your shaders, but this is my framework which is all about experimentation (and I don't have a deadline so I'm able to invest time).
Right now it's a trade off between flexibility vs performance vs artist-friendly.
In a perfect scenario, I would rather not have any hard coded hlsl/glsl/cg in the graph to make it fully API independent (and emulatable on a fixed-function system). I see complex algorithms leading to spaghetti graphs (an entire graph could be encapsulated inside a single node - much like a function call). The main issue I see with this system though is performance wise from the code generation (no hand coded optimisations). It is artist friendly (to technical artists) without having to write in a shader language, but with enough "function nodes" you could abstract away the complexity.
The most performance-ideal version of the system is to have huge do-it-all nodes hardcoded in CG. e.g. a node called DiffuseAmbientSpecularReflectionNormalMap that takes a Diffuse colour value, an Ambient colour value, specular values, a reflection value, a normal, which you can either connect a texture sampler to or a hard float. This is also the most artist friendly since it's also simple to understand.
The most artist friendly version would be to imitate Maya's or Max's material system. But I have seen some really huge materials made with this that it'd be a performance issue.
I recently implemented a shader graph editor and run-time code generator, and I'm extremely happy with how it turned out. You can read more about it here:
http://www.terathon.com/wiki/index.php?title=Using_the_Shader_Editor
It uses a lot of the same ideas that you mentioned in your first post. It runs on PC, Mac, and Playstation 3, and it outputs GLSL, Cg, or ARB fragment program code as necessary, depending on the underlying graphics hardware and platform. There are simple nodes for basic mathematical operations and more complex nodes that perform common tasks like computing Blinn specular reflection. Shaders are generated on the fly for a variety of possible lighting environments (3 ambient environments, 5 types of light source, and 3 types of fog). The system differs from your design in that you don't have explicit control over the vertex program. The vertex program is generated automatically based on what interpolants are used in the pixel shader.
The shader editor has already been released to C4 Engine licensees, and we'll be posting a free demo in the next week or so if you'd like to play with it.
http://www.terathon.com/wiki/index.php?title=Using_the_Shader_Editor
It uses a lot of the same ideas that you mentioned in your first post. It runs on PC, Mac, and Playstation 3, and it outputs GLSL, Cg, or ARB fragment program code as necessary, depending on the underlying graphics hardware and platform. There are simple nodes for basic mathematical operations and more complex nodes that perform common tasks like computing Blinn specular reflection. Shaders are generated on the fly for a variety of possible lighting environments (3 ambient environments, 5 types of light source, and 3 types of fog). The system differs from your design in that you don't have explicit control over the vertex program. The vertex program is generated automatically based on what interpolants are used in the pixel shader.
The shader editor has already been released to C4 Engine licensees, and we'll be posting a free demo in the next week or so if you'd like to play with it.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
