Sign in to follow this  
AvengerDr

Unity Yet another shader generation approach

Recommended Posts

AvengerDr    751
In the past few months I found myself juggling between different projects  aimed at several different platforms (Windows 7,8, RT and Phone). Some of those have different capabilities so some shaders needed to be modified in order to work correctly. I know that premature optimization is just as bad but in this specific situation I thought that addressing the problem sooner than later would be the right choice.
 
To address this problem I created a little tool that allows me to dynamically generate a set of shaders through a graph like structure. Which is nothing new, as it is usually the basis for this kind of application. I did probably reinvent a lot of wheels but since I couldn't use MS's shader designer (it only works with C++ I think) nor Unity's equivalent (as I have my own puny little engine) I decided to roll my own. I am writing here to get some feedback on my architecture and if there is something I overlooked.
 
Basically I have defined classes for most of the HLSL language. Then there are nodes such as constants, math operations and special function nodes. The latter are the most important ones as they correspond to high-level functions such as Phong Lighting, shadow algorithms and so on. Each of these function nodes expose several switches that enable me to enable/disable specific features. For example if I set a Phong node's "Shadows" to true then it will generate a different signature for the function than if it had it set to false. Once the structure is complete the graph is traversed and the actual shader code is generated line by line. From my understanding I think that dynamic shader linking works similarly but I've not been able to find a lot of information on the subject.
 
Right now shaders can only be defined in code, in the future I could build a graphical engine. A classic phong lighting pixel shader looks like this and this is the generated output. It is also possible to configure the amount of "verbosity". The interesting thing is that once the shader is compiled it gets serialized to a custom format that contains other information. Variables and nodes that are part of the shader are decorated with engine references. If I add a reference to the camera position for example, that variable tells the engine that it has to look for that value when initialising the shader. Analogously for the values needed to assemble constant buffers (like world/view/projection matrices). 

Once the shader is serialised, all this metadata helps the engine to automatically assign each shader variable or cbuffer with the right values. Before in my engine, each shader class had these huge parts of code that fetched needed values from somewhere else in the engine. Now all that has been deleted and is taken care automatically as long as the shaders are loaded in this format.
 
[attachment=18541:ShaderGenV01.png]

Another neat feature is that within the tool I have built I can define different techniques; i.e: a regular Phong shader, one using a Diffuse Map, one using a Shadow Map. Each technique maps a different combination of vertex and pixel shaders. The decoration that I mentioned earlier helps the tool generate a "TechniqueKey" for each shader that is then used by the engine to fetch the right shader from the file on disk. For example the PhongDiffusePS shader is decorated with attributes defining its use of a DiffuseMap (among other things). When in the actual application I enable the DiffuseMap feature, the shader system checks whether that feature is supported by the current set of shaders assigned the material. If a suitable technique is found, then the systeme enables the relevant parameters. In this way it is also possible to check for different feature levels and so on.

Probably something like this is overkill for a lot of small projects and I reckon it is not as easy to fix something in the generated code of this tool as it is when making changes in the actual source code it self. But once it does finally work, the the automatic configuration of shader variables is something that I like (at least if compared to my previous implementation, I don't how everyone else handles that). What I am asking is how extensible or expandable this approach is (maybe it is too late to ask this kind of questions biggrin.png)? Right now I have a handful of shaders defined in the system. If you had a look at the code, what kind of problems am I likely to run into when adding nodes to deal with Geometry shaders and other advanced features?

Finally, if anyone could be interested in having a look at the tool I'm happy to share it on GitHub. Edited by AvengerDr

Share this post


Link to post
Share on other sites
AvengerDr    751

Sure, it is available here. You only need the librariers named "Odyssey.Common" and "Odyssey.Content". The rest is part of another project, a 2D UI library. The tool itself is the project named ShaderGenerator.

 

In order to build it you will probably have to change the SharpDX SDK line in the Build/ ... targets file to the location you have installed it. The two above projects should be compiled using the Win8Debug configuration. It should work on W7 too. The shaders themselves are defined in the ShaderGenerator/Shaders/Techniques subfolder. The Data/ResourceManager.cs class defines which shaders are loaded at startup. Though saving and loading shader graphs should work. You will need the ColorCode and Galasoft.MvvmLight nuget packages installed.

 

Let me know if you are able to run it. In order to compile shaders every item listed must be assigned to a technique. At the moment these are hardcoded but I am in the process of adding a new window that will allow defining new techniques.

 

Each technique can be decorated in code by Vertex/PixelShaderAttribute(s). These attributes are used when validating nodes. All attributes defined will be used to validate nodes that have the same attribute attached. For example the PhongPS shader does not have the PixelShaderAttribute for DiffuseMap. Thus when validating the PhongLightingNode, the DiffuseMapSampler node won't be checked and so on.

 

Clearly in order to use the automatic configuration of shader variables you would need to parse the metadata in the shader collection file from your own engine. I can post some examples if needed.

Edited by AvengerDr

Share this post


Link to post
Share on other sites
AvengerDr    751

Hi all, I've added support to regular HLSL source code files. Now it can act as a graphical frontend for offline shader compilation. Just press the "import" button and select .hlsl source code files. You can find a link to an installer from here. In the post you'll find also more detailed information regarding the shader graph structure, which now supports Yaml (an example is included in the installer).

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this  

  • Similar Content

    • By dobbey
      Who are you?
      We call ourselves Dobbey. Our team currently consists of two people.
      We are both early 20's and come from Germany. At the moment this is our hobby.
      What's the name of the game and what is it about?
      The game is called Insane and it is a 2D adventure puzzle game with horror elements.
      Insane is about a scientist who has been doing human experiments for years.
      His most successful experiment is number 73, which seems to be mentally labile.
      This experiment manages to flee one day but can't distinguish between reality and imagination.
      In this game you will find a creepy atmosphere with difficult and varied puzzles.
      What inspired you?
      The style of the game is inspired by Limbo. But we were also inspired by Jump 'n' Run games and movies.
      Of course, we have also our own elements like an inventory system, butterfly effect, the "INSANE" effect, using a QR-Code..
      Which program do you use to develop the game?
      We are programming it with Unity. In our opinion, it is the best program for indie games.
      And of course we are working with Photoshop to create the graphics.
      What platform will Insane be available on?
      For PC.
      How can I support you?
      Twitter:        http://bit.ly/20POkfM    
      YouTube:     http://bit.ly/2rLXoM0
      Instagram:   http://bit.ly/2qwU2bl
      Facebook:    http://bit.ly/2p3nwRQ
      Website:       http://bit.ly/2s8buqU
       

      Here you can see some of our screenshots.

       

      (You can find the animation on Twitter/Instagram.)
       

       

       

      (You can find the animation on YouTube.)

      What do you think about it?
       
      Yo can see more screenshots or videos on Twitter, Instagram or YouTube.
    • By Liquid1Phantom
      Hello, my name is Thomas and I am currently starting up a project. 
      The project is going to be a very hardcore, and tactical FPS that is similar to Escape from Tarkov. It will be made within Unity, and will be programmed in C#. Currently, I have a Unity project started, an organized Discord setup, and as well as a very organized Google Drive. The Google Drive has art references, and plenty of folders for organization. Lastly, the payment for this project will be solely rev-share as I have no money.
      I am looking for anyone who can contribute: 2D artists, 3D artists, composers, programmers, etc.
      If you would like to help out and be of use, please email me at: thomasmunson2277@gmail.com
      Alternatively, add my Discord: Thomas#3788
    • By ilovegames
      Score points by shooting enemies! Before you is one of the best representatives of the simulator sniper games genre, with stunning graphics, quality effects, and realistic ballistics of bullets. It will not be easy to win for sure. Are you ready? Then take a sniper rifle and fight!
      Download http://falcoware.com/Sniper.php




  • Popular Now